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Abstract 


Currently,  e-  learning  technology  is  being  used  to  train  and  educate  a  myriad  of 
personnel  and  each  year  enrollment  numbers  grow.  Evidence  shows,  however,  that 
completion  rates  among  e- learners  are  lower  than  that  of  traditional  learners. 

Motivational  theory  is  applied  to  this  problem  to  explain  why  e-learners  initiate,  sustain, 
and  terminate  behavior.  In  particular,  an  integrative  motivational  model,  that  highlights 
distal  and  proximal  processes,  is  introduced  to  identify  and  measure  those  factors  most 
likely  to  influence  e-learning  course  completion  rates.  Findings  offer  recommendations 
that  may  be  useful  to  e-  learning  course  instructors,  administrators,  and  designers. 

Three  research  questions,  guided  by  13  hypotheses,  were  used  to  investigate 
motivational  theory  and  its  relation  to  e-learning  course  completion  rates.  Eight  (8)  e- 
leaming  courses  were  analyzed  along  with  497  responses  received  from  an  on-  line 
survey.  Data  was  coded  according  to  whether  the  student  completed  or  dropped  the 
course.  Statistical  analysis  showed  that  e- learners  are  more  likely  to  invest  their  time, 
talent,  and  energy  when  they  encounter  fewer  technical  problems,  fewer  distractions,  and 
more  environmental  support  from  supervisors  and  instructors.  Furthermore,  lengthy 
modules  and  low  self-efficacy  were  found  to  decrease  the  motivational  tendency  to 
persist.  Overall,  results  demonstrated  that  motivational  theory  can  be  used  to  predict  and 
explain  those  factors  most  likely  to  influence  a  person’s  desire  to  “go  the  distance”  with 
e-learning.  Practical  and  theoretical  implications  of  the  research  are  discussed. 
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GOING  THE  DISTANCE  WITH  DISTANCE  LEARNING:  AN  ANALYSIS  OF 


MOTIVATIONAL  FACTORS  THAT  INFLUENCE  E-LEARNING  COURSE 

COMPLETION  RATES 


1.  Introduction 

“Even  as  we  use  technology  to  shape  our  environments,  technology  is  shaping  us.  ” 

Michael  Schrage  (1995) 


Background 

The  rapid  expansion  of  the  Internet  has  promoted  the  neeessity  for  teehnologieal 
competence  as  well  as  promised  the  global  connectivity  of  educational  material  for 
distance  learning  (Fabos  &  Young,  1999;  Fetterman,  1998).  Many  edueators  eompare 
the  Internet  revolution  with  the  California  gold  rush  of  the  mid  1 800s.  The  educational 
hype  brought  by  this  new  technology  has  often  left  unfulfilled  promises  (Howell,  2001). 
Studies  reveal  that  distance  learners  using  the  Internet  as  their  educational  platform  tend 
to  have  lower  course  completion  rates  than  traditional  classroom  learners  (Cheng, 
Lehman,  &  Armstrong,  1991;  Jewett,  1997;  Phelps,  Wells,  Ashworth,  &  Hahn,  1991; 
Phipps  &  Merisotis,  1999),  sometimes  by  more  than  40  percent  (Zielinski,  2000).  In  one 
study,  while  the  on-campus  course  completion  rate  was  85  percent,  only  66  percent  of  the 
students  successfully  completed  the  same  course  via  the  Internet  (Jewett,  1997). 
Unfortunately,  these  studies  do  not  adequately  explain  why  Internet  based  course 
completion  rates  are  lower.  As  stated  by  one  electronic  learning  (e-  learning)  course 
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designer,  “Completion  rates  determine  e- learning  success,  but  I  am  not  sure  what  it  is  that 
motivates  people  to  complete  them”  (Miller,  2001). 

The  proliferation  of  e-  learning  technologies  has  created  a  need  to  understand 
factors  that  decrease,  as  well  as  increase  their  optimal  use.  A  look  at  factors  that 
influence  e- learning  course  completion  rates  is  a  step  in  this  direction.  This  study  focuses 
on  the  role  motivation  plays  in  explaining  why  some  people  persist  and  others  drop  out  of 
e-learning  courses.  In  particular,  the  study  will  seek  to  identify  and  measure  those 
motivational  factors  that  promote  or  inhibit  e-  learning  course  completion  rates. 

E- learning  is  a  subset  of  distance  learning — a  generic  term  for  experiencing 
learning  in  some  form  other  than  traditional  instructor-  led  training  in  the  physical 
presence  of  students.  Distance  learning  includes  correspondence  courses  delivered 
through  the  mail  as  well  as  courses  delivered  electronically  through  satellite  broadcasts, 
videotapes,  video  conferencing,  and  computer-based  training.  E-  learning,  on  the  other 
hand,  is  education  delivered  by  computer  via  a  network.  The  network  is  most  often  the 
Internet,  but  could  also  include  an  intranet  or  local  area  network. 

E-  learning  is  an  innovative  concept  that  educates  and  trains  personnel  using  a 
presentation  rich  format  to  deliver  information  and  instruction  across  a  network  directly 
to  a  desktop  or  laptop  computer.  In  contrast  to  traditional  classroom  instruction  where 
teachers  and  students  interact  directly,  teachers  and  students  interact  indirectly  via  phone, 
e-mail,  or  web-based  chat  rooms  in  e-  learning  courses.  While  computer  based  training 
systems  are  also  technology-based  learning  systems,  they  do  not  have  the  same  level  of 
flexibility  to  update  and  conduct  education  and  training  any  time  and  anywhere  like  e- 
leaming  does. 
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Some  e-learning  researchers  have  indicated  that  motivation  plays  a  key  role  in 
determining  human  behavior  in  learning  environments,  and  could  be  the  underlying  cause 
of  e- learning’s  relatively  low  completion  rates  (Dalton,  Manning,  Hagen,  Paul,  &  Tong, 
2000;  Finnemann,  1998;  Hellebrandt,  1999;  Hoffman,  1995;  Lee,  1997).  Up  until  now, 
very  little  evidence  has  been  collected  to  support  this  hypothesis.  This  research  addresses 
this  deficiency  by  using  motivation  theory  to  examine  what  influences  an  e- learner’s 
desire  to  “go  the  distance”  with  e-  learning.  An  investigation  using  motivation  constructs 
may  lead  to  a  better  understanding  of  how  e-  learning  course  design  can  promote  or 
inhibit  course  completion.  Practitioners,  academics,  and  researchers  can  use  this 
information  to  devise  practical  methods  for  evaluating  e- learning’s  effectiveness  as  well 
as  design  motivational  features  into  e- learning  courses  to  improve  completion  rates. 

The  Concept  of  E-learning 

E- learning’s  origins  date  back  at  least  to  1984  and  the  advent  of  computer-based 
training  courses  delivered  via  floppy  disk.  As  the  World  Wide  Web  evolved  during  the 
mid-  to  late  1990’s,  training  providers  began  to  explore  new  ways  to  impart  education 
and  training  by  taking  what  was  in  print  and  delivering  it  online.  These  online 
connections  produced  a  virtual  learning  environment  where  students  and  teachers  were 
now  able  to  interact  indirectly  through  hardware  and  software.  Online  connections  also 
provided  students  with  access  to  a  much  broader  range  of  educational  and  training 
material. 

In  its  simplest  form,  e- learning  is  “...the  use  of  Internet  technologies  to  deliver  a 
broad  array  of  solutions  that  enhance  knowledge  and  performance”  (Rosenberg, 
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2001:28).  In  most  instances,  participants  connect  to  e-learning  systems  via  the  Internet  or 
company  intranet  using  a  logon- id  and  password.  E- learning  then  provides  the  training 
and  instruction  by  using  various  combinations  of  text,  graphics,  animation,  sounds, 
streaming  video,  hotlinks,  flipbooks  (viewer  controlled  mini  movie),  and  self-running 
screen  capture  display  programs  (automated  slideshow)  to  further  enhance  the 
experience.  All  of  these  features  are  used  in  conjunction  with  a  computer  system  to 
convey  the  required  information  and  provide  training  that  is  much  more  appealing  than 
the  formal  and  static  nature  of  text-based  training  (Mayor,  2001:1).  The  concept  of  a 
typical  e- learning  environment  is  diagramed  in  Figure  1. 


As  can  be  seen  from  the  diagram,  users  interact  directly  with  technology  to 
receive  the  type  of  learning  they  need  or  desire.  Both,  the  quality  of  the  interface 
between  users  and  the  technology,  and  the  design  of  e-learning  courses  play  a  major  role 
in  influencing  the  effectiveness  of  the  learning  experience.  Users  must  also  contend  with 
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factors  like  goals,  demands,  and  constraints  (e.g.,  hardware  and  software  limitations, 
security  issues)  that  may  interrupt  the  learning  process  or  hinder  the  user’s  desire  to 
continue. 

Today,  the  Internet,  and  similar  network-based  systems  fulfill  a  rapidly  growing 
demand  for  distance  learning  with  their  great  accessibility  and  flexibility.  E-  learning 
technology,  in  particular,  continues  to  evolve  as  a  premier  way  to  educate  and  train  the 
workforce  (Abell,  2001;  Dalton  &  others,  2000;  Hall,  2000;  Katz  &  Oblinger,  2000; 
Mayor,  2001;  Rosenberg,  2001).  The  private  and  academic  sectors  have  embraced  this 
technology  to  efficiently  disseminate  education  and  training,  as  well  as  to  maintain  a 
competitive  market  edge.  In  the  United  States,  e-  learning  is  the  fastest  growing  market 
segment  of  adult  education  (Carr-Chellman,  Fitzpatrick,  Ke  Zhang,  &  Salt,  2000:291). 
Last  year  American  schools  and  corporations  poured  more  than  $  1  billion  into  this  new 
technology,  and  leading  researchers  of  distance  learning  technology  project  that  figure 
will  surpass  $10  billion  by  the  year  2003  (Dobbs,  2001:24;  Grimes,  2001  :R6). 

The  future  of  e- learning  in  the  federal  government  looks  bright  as  well.  White 
House  Executive  Order  #13111  (1999)  requires  each  executive  department  to  choose  one 
area  of  training  and  implement  some  sort  of  e-learning  initiative.  The  purpose  of  this 
order  is  to  encourage  the  effective  use  of  technology  to  improve  training  opportunities  for 
federal  government  employees. 

The  Department  of  Defense  (DoD)  has  also  adopted  e- learning  as  a  means  to 
educate  and  train  its  personnel.  E-  learning  appeals  to  the  military  because  it  eliminates 
the  barriers  of  place  and  time  by  providing  education  and  training  anywhere  in  the  world, 
at  a  fraction  of  “in-person”  instructor  costs  (DoD  1993  Bottom-Up  Review;  Rosenberg, 
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2001:26).  For  example,  the  DoD’s  Readiness  and  Training  unit,  stationed  at  the 
Pentagon,  has  implemented  e-learning  as  a  means  to  teach  officers  how  to  employ  joint 
force  military  doctrine  in  battle  and  other  situations  (Klaila,  2001).  In  addition,  the  Army 
has  launched  Army  University  Online,  which  allows  soldiers  to  participate  in  e-  learning 
courses  to  continue  their  education  and  training  no  matter  where  they  are  deployed  in  the 
world  (Seffers,  2001).  E-leaming  has  become  integral,  if  not  essential,  to  the  operation 
of  modern-day  learning  in  military  organizations. 

The  Air  Force  Institute  of  Technology  School  of  Systems  and  Logistics,  at 
Wright-Patterson  Air  Force  Base,  Ohio,  is  exploring  ways  to  build  motivational  features 
into  e-leaming  that  help  improve  completion  rates  for  their  Virtual  Schoolhouse  courses. 
The  Virtual  Schoolhouse  (VSH)  is  a  professional,  flexible,  continuing  education  resource 
that  offers  instruction  in  acquisition  fundamentals  by  means  of  the  worldwide  web 
(AFIT/LS  Department  of  Systems  Acquisition  Management  Homepage,  2002).  So  far, 
while  e-learning  has  proven  to  be  a  promising  concept,  VSH  administrators  believe  that 
the  completion  rates  for  several  of  their  courses  are  lower  than  that  of  traditional 
classroom  courses  and,  therefore,  must  be  improved  upon. 

Implications 

As  Internet  technology  continues  to  expand,  it  is  likely  that  the  number  of  e- 
leaming  courses  will  also  increase.  A  pivotal  factor  in  maximizing  the  power  of  this 
technology  becomes  our  ability  to  create  courses  that  preserve  an  e- learner’s  desire  to 
persist  until  completion.  If  a  substantial  number  of  distance  learners  fail  to  complete 
their  courses,  then  the  notion  of  unlimited  access  to  information  and  instmction  becomes 
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meaningless  (Phipps  &  Merisotis,  1999:25).  Identifying  the  appropriate  motivational 
factors  that  influence  course  completion  rates  has  proven  to  be  challenging  (Rosenberg, 
2001:42).  E-learning  is  an  entirely  different  style  of  education  that  is  not  yet  fully 
understood  (Phipps  &  Merisotis,  1999:1).  Misguided  perceptions,  about  e- learning’s 
ability  to  train  and  educate  people,  have  done  more  to  cloud  its  usefulness  than  to 
enhance  the  learning  experience  (Cox,  1999).  Being  able  to  identify  those  motivational 
factors  that  inhibit  or  promote  e- learning  course  completion  rates  may  help  un- cloud  its 
usefulness. 

Motivational  Theory  Approach 

Though  no  one  approach  is  likely  to  capture  all  the  dynamics  involved  in 
determining  whether  or  not  people  will  successfully  complete  an  e-  learning  course, 
motivational  theory  provides  readily  available  constructs  that  may  help  explain  why 
students  invest  their  time,  talent,  and  energy  in  e-  learning  educational  and  training 
opportunities  (Maehr  &  Braskamp,  1986:17).  Schunk  defined  motivation  as  “the  process 
whereby  goal-directed  behavior  is  instigated  and  sustained”  (1990:3).  This  study  makes 
use  of  Schunk’s  definition  of  motivation,  and  considers  process  factors  that  direct, 
intensify,  and  terminate  goal-directed  behavior  (Campbell  &  Pritchard,  1974). 
Understanding  the  motivational  process  may  help  provide  insight  as  to  why  some 
individuals  complete  their  courses,  while  others  drop  out.  Motivational  theory  provides 
guidance  for  determining  those  factors  most  likely  to  influence  a  person’s  desire  to 
complete  an  e-  learning  course. 
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Research  Focus 


In  recent  years,  there  has  been  considerable  effort  to  research  the  costs, 
implementation,  and  return  on  investment  issues  of  e-learning  (Rosenberg,  2001:48). 
Some  of  this  research  examined  the  synchronous  or  asynchronous  features  of  e-learning 
and  measured  interactivity  factors  like  download  speeds,  collaboration  capability  levels, 
and  multimedia  use  (Sevcik,  1996:27).  Other  studies  examined  its  effectiveness  by 
comparing  it  to  traditional,  brick  and  mortar  classroom  methods  of  learning.  This  study 
takes  a  different  approach  by  focusing  on  a  student’s  motivation  to  complete  e- learning 
courses  based  on  the  amalgamate  of  external  factors  such  as  technical  problems,  off-task 
demands,  and  environmental  support,  with  motivational  factors  and  course  design 
characteristics.  The  study  plans  to  look  at  motivational  processes  and  make  comparisons 
across  e-  learning  courses  and  across  students  to  explain  any  variation  in  course 
completion  rates. 

Problem  Statement 

E-learning  is  being  used  to  train  and  educate  a  myriad  of  Air  Force  personnel  and 
each  year  e-  learning  enrollment  numbers  grow.  Evidence  shows,  however,  that 
completion  rates  among  e- learners  are  lower  than  that  of  traditional  learners. 
Unfortunately,  there  are  no  definitive  answers  as  to  why.  This  dictates  the  need  to 
explore  those  factors  that  influence  e-  learning  course  completion  rates.  Efforts  may  in 
turn  allow  organizations,  like  the  Air  Force  Institute  of  Technology  School  of  Systems 
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and  Logistics,  to  design  motivationally  sound  courses  that  better  harness  e- learning’s 
capability. 

Transcripts  from  personal  interviews  with  nine  e- learning  administrators  were 
used  to  create  the  focus  of  this  research.  Interview  questions  were  designed  to  extract 
information  on  perceptions  as  to  why  some  individuals  complete  e-  learning  courses, 
while  others  drop  out.  The  questions  were  open-ended  towards  the  beginning  of  the 
fieldwork,  but  became  more  structured  as  the  research  effort  evolved.  The  general 
findings  from  the  interviews  led  to  the  model  depicted  in  Figure  2. 


Figure  2:  General  Findings  from  Interview  Transcripts 

As  can  be  seen  from  the  model,  motivation,  or  lack  thereof,  was  viewed  as  a  key 
determinant  of  e- learning  course  completion  rates.  When  asked  to  identify  e- learning 
factors  that  influenced  this  motivation,  the  e- learning  administrators  identified  three 
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external  factors:  technical  limitations  or  problems  with  computer  hardware  and  software, 
off- task  demands  encountered  while  engaged  with  e- learning,  and  the  amount  of  support 


received  from  the  environment  that  promotes  the  e-  learning  effort.  They  also  believed 
that  course  design  can  lessen  the  negative  effects  and  enhance  the  positive  effects  of  these 
external  factors  and,  in  turn,  motivational  constructs  on  e-learning  course  completion 
rates. 


Technical  Problems 

Several  of  the  administrators  stated  that  e-  learners  who  continuously  experience 
technical  problems  like  slow  or  choppy  response  times  are  more  likely  to  withdraw  from 
e-learning.  One  administrator  summed  up  the  technical  aspects  of  e- learning,  by  stating, 
“We  either  lose  students  on  the  high-end  or  low-end.  E-learning  courses  full  of 
innovative  bells  and  whistles  (high-end)  are  ineffective  if  students  are  limited  by  firewalls 
or  do  not  have  the  proper  hardware  or  software  to  handle  them.  On  the  other  hand, 
courses  that  are  static  and  boring  (low-end)  do  not  adequately  grab  and  keep  the  e- 
learner’s  attention”  (Lewis,  2001). 

Course  Design  and  Technical  Problems 

Elements  of  course  design  can  accentuate  or  dampen  the  effects  of  technical 
problems  on  e- learners’  motivation.  One  e- learning  designer  noted,  “Obtaining  high  e- 
leaming  completion  rates  hinge  on  designing  courses  that  provide  adequate  amounts  of 
interaction  and  feedback,  and  a  set  time  limit  in  which  the  course  must  be  finished” 
(Miller,  2001).  Implementing  these  features,  he  believed,  would  increase  an  e- learner’s 
commitment  and  motivation  because  it  forces  them  to  interact,  be  attentive,  and  get  the 
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course  done  in  a  timely  manner.  However,  he  added,  “It  is  important  for  designers  to 
know  their  customers  limitations  so  a  proper  balance  can  be  reached  between  the 
practical  and  impossible.”  There  were  similar  findings  within  e- learning  course  critiques. 
The  negative  comments  most  frequently  made  were  centered  around  technical  or  content 
problems  with  the  course.  Course  critiques  identified  such  factors  as  slow  system 
response  times  and  poor  or  wordy  course  content  as  areas  of  concern.  Un- compelling, 
static  content  (low-end)  also  serves  as  a  motivational  roadblock  because  it  reduces 
interaction  to  simple  reading  (Dalton  &  others,  2000:5). 

Off-task  Demands 

Off-task  demands  were  another  area  of  concern  for  the  interviewees.  Most  of  the 
administrators  reported  that  e-  learning  is  often  conducted  in  an  uncontrolled  or  work- 
related  environment  in  which  the  e- learner  is  susceptible  to  distractions  and  interruptions. 
Off-tasks  hinder  the  e-  learning  effort  because  they  force  the  e-  learner  to  allocate  time, 
energy,  and  attention  towards  things  not  related  to  completing  the  course  at  hand.  As  put 
by  one  e- learning  administrator,  “To  be  successful  at  e- learning  the  environment  must 
allow  the  student  to  devote  the  necessary  time,  attention,  and  energy  to  the  course.”  The 
administrator  went  on  to  say  that,  “...in  some  cases,  e- learners  should  be  removed  or 
remove  themselves  from  these  uncontrolled  environments”  (Gaudreault,  2001).  Many 
comments  from  the  e- learning  critiques  stated,  “I  had  difficulty  completing  the  e- learning 
course  with  my  current  workload  and  the  many  distractions  I  encountered,”  or  “My  job 
sent  me  TDY  mid- way  through  the  course.”  Observations  found  that  competing  demands 
complicated  the  motivational  process  and  “delayed”  efforts  to  complete  the  course. 
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Frequent  interruptions  from  peers,  email,  and  telephone  calls  force  students  to  reallocate 
their  attention  from  the  e-  learning  course. 

Course  Design  and  Ojf-task  Demands 

Elements  of  course  design  can  also  accentuate  or  dampen  the  effects  of  off-task 
demands  on  e- learners’  motivation.  Course  structures  that  provide  adequate  and  timely 
feedback  help  e- learners  maintain  focus  on  completing  the  goals  and  objectives. 
Observation  notes  revealed  that  high  levels  of  support  like  feedback  helped  re-  focus 
efforts  and  increase  the  levels  of  persistence  towards  course  completion.  Students  often 
report  a  desire  to  simply  “printout”  the  e-learning  course  material  and  go  read  it  in  a 
quiet,  environment- friendly  place.  Difficulty  printing  e- learning  course  material  is 
another  common  criticism  from  the  course  critiques. 

Environmental  Support 

Environmental  support  is  the  third  area  of  concern  for  the  e- learning 
administrators.  They  believed  that  the  amount  of  support  e-  learners  receive  from 
supervisors,  peers,  instructors,  and  family  members,  goes  a  long  way  in  determining  the 
success  of  their  e- learning  experience.  Support  includes,  but  is  not  limited  to,  being 
given:  the  proper  resources  to  conduct  e-  learning,  the  time  to  devote  attention  and  energy 
towards  the  course  without  disruptions,  and  the  opportunity  to  take  the  course  for  career 
advancement.  Critique  comments  that  were  negative  in  nature  stated  such  things  as,  “My 
boss  or  co-workers  dislike  e-leaming — they  thought  I  was  slacking  off,”  or  “I  received 
low  amounts  of  course  feedback  and  interaction  with  my  instructor.”  Positive  critiques 
included  statements  like,  “It  was  nice  to  be  given  the  time  to  devote  to  e-learning  so  that  I 
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could  actually  work  at  my  own  pace,”  and  “The  outstanding  support  I  received  from  the 
e- learning  administrators  made  a  difference.” 

Research  Questions 

Analysis  of  the  initial  interviews  and  observations  led  to  the  following  research 
questions  eoneeming  external  factors  and  course  design. 

Research  Question  1 :  In  what  ways  do  technical  problems,  off-task  demands, 
and  environmental  support  (external  factors)  influence  motivation  to  complete  e- learning 
eourses? 

Research  Question  2:  How  does  e-learning  course  design  influence  the  effects  of 
external  factors  on  motivational  constructs? 

The  interviews  and  observations  produced  a  fairly  clear  picture  of  the  relative 
influence  of  the  three  external  factor  and  moderating  effect  of  course  design  on 
completion  rates,  but  provided  less  insight  into  how  these  factors  influenced  motivational 
processes.  Further  research  into  the  motivational  theory  was  required  to  identify  the 
specific  processes  that  may  be  influenced  by  external  factors  and  e- learning  course 
design. 

Research  Question  3 :  What  motivational  factors  influence  e-  learning  course 
completion  rates? 
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Thesis  Overview 


This  chapter  briefly  introduced  the  problems  faced  by  e- learners  and  e- learning 
course  administrative  in  terms  of  external  and  design  factors  that  influence  course 
completion  rates.  The  chapter  also  builds  a  case  for  motivational  theory  as  a  possible 
approach  to  explain  disappointing  e-  learning  course  completion  rates.  Chapter  II  presents 
a  review  of  previous  research  on  motivational  theory  and  other  literature  relevant  to  the 
research.  Chapter  III  presents  the  methodology  used  to  answer  the  research  question  and 
describes  the  course  selection,  research  instrument  development,  and  data  collection 
techniques.  Chapter  IV  details  the  results  of  the  research  and  provides  the  statistical 
analysis  of  the  data.  Chapter  V  presents  the  conclusion  and  recommendations  from  this 
research. 
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II.  Literature  Review 


“It  is  the  spirit  that  motivates,  that  calls  upon  a  man ’s  reserves  of  dedication  and 
effort,  that  decides  whether  he  will  give  his  best  or  just  enough  to  get  by.  ” 

Peter  Drucker  (1954) 


Introduction 

This  chapter  reviews  and  then  integrates  motivation  theory  to  identify  factors  that 
influence  completion  rates  in  e- learning  environments.  The  majority  of  this  literature  is 
taken  from  the  field  of  psychology’s  sub-disciplines  of  motivational  and  cognitive 
psychology,  with  a  few  sources  being  cited  from  research  in  the  organizational  and 
educational  domains.  This  chapter  is  divided  into  several  sections  presenting  a  road  map 
of  the  external  factors  and  motivational  constructs  people  encounter  while  engaged  in  e- 
leaming  activities. 

The  chapter  starts  with  a  review  of  several  attrition  studies  and  how  their  findings, 
combined  with  the  external  factors  introduced  in  chapter  one,  influence  completion  rates 
in  e-learning  environments.  Next,  the  motivational  process  model  is  presented  followed 
by  an  explanation  of  distal  and  proximal  concepts  of  motivation.  Several  motivational 
theories  are  then  used  to  clarify  how  behavior  is  initiated,  directed,  intensified  and 
sustained.  This  chapter  concludes  with  a  discussion  of  an  integrative  approach  to 
motivation.  By  applying  a  combination  of  explanatory  text  and  literature  on  current 
theories  of  motivation,  scholars  and  researchers  may  obtain  a  more  comprehensive 
understanding  of  those  factors  that  influence  e- learning  course  completion  rates. 
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External  Factors 


Theories  have  been  used  to  explain  why  people  persist  or  drop  out  of  learning 
environments  (Catalano,  1985;  Miller,  1967;  Spanard,  1990).  Miller  (1967)  proposed  a 
push-pull  theory  in  which  positive  driving  forces  push  an  adult  toward  completion  while 
negative,  restraining  factors  pull  the  person  away.  The  theory  indicates  that  retention 
depends  on  the  degree  of  congruence  or  conflict  between  the  person’s  needs  and  the 
perceived  strength  of  the  social  and  situational  factors  in  the  decision  (Miller,  1967). 

This  theory  is  similar  to  Lewin’s  (1951)  theory  of  Force-Field  Analysis,  which  states  that 
human  behavior  is  a  result  of  competition  between  driving  and  restraining  forces. 

Catalano  (1985)  developed  a  similar  model  to  explain  retention  among  students  in 
college.  The  model  helps  define  and  organize  the  complexity  of  students’  choices  to 
remain  in  college  as  it  relates  to  the  pushing  and  pulling  forces  they  encounter.  In 
addition,  Catalano’s  model  adds  the  non- educational  aspects  of  the  person’s  life  by 
combining  the  salience  of  all  forces  drawing  on  a  student’s  attention  and  energy 
simultaneously  with  the  costs  and  benefits  of  education  (Spanard,  1990:323). 

Spanard  (1990)  proposed  a  descriptive  model  illustrating  the  path  of  adult 
problem  solving  and  thinking  that  leads  to  retention  and  eventual  completion  of  academic 
programs.  Spanard  introduced  several  theories,  including  Catalano’s  and  Miller’s,  in  her 
study.  She  presents  a  longitudinal  model  that  suggests  many  factors  weigh  in  the 
decision  to  stay  or  drop  out  of  programs  (Spanard,  1990:309).  Of  these  factors,  a 
person’s  aspiration,  effort,  and  persistence  to  achieve  the  goal  of  program  completion  are 
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important  elements  in  each  stage  of  the  “stay  or  leave”  decision-making  process 
(Spanard,  1990:309). 

Though  much  of  the  early  retention  research  focused  on  adult  learners  enrolled  in 
traditional  classroom  settings,  several  of  its  findings  could  possibly  apply  to  e-  learning 
environments.  Lower  levels  of  self-motivation  and  career  aspiration,  combined  with 
uncertain  goals,  inadequate  work  habits,  and  immature  attitudes  and  perceptions,  were 
associated  with  an  increased  likeliness  to  drop  out  of  school  (Hoyt,  1999;  Morrison, 

1999;  Spanard,  1990;  Waxman  &  Huang,  1996).  The  research  also  suggests  that  the 
lower  the  amount  of  environmental  support  (e.g.,  family,  instructor,  job,  social),  the  more 
likely  the  student  will  drop  out  of  school  (Hoyt,  1999;  Waxman  &  Huang,  1996). 

Findings  from  this  research  have  provided  valuable  information  that  practitioners  have 
used  to  develop  better  retention  programs  (Morrison,  1999:1 1;  Spanard,  1990:309). 

The  external  factors  introduced  in  Chapter  1  are  presented  in  this  research  to 
identify  those  pushing  and  pulling  forces  that  influence  completion  rates  within  e- 
leaming  environments.  E-learning  environments  present  some  additional  challenges  not 
found  by  students  in  traditional  classroom  settings.  For  starters,  traditional  classroom 
learning  typically  takes  place  in  a  controlled  environment  in  which  external  distractions 
are  unlikely.  E-  learners,  on  the  other  hand,  often  contend  with  external  factors  like 
network  problems,  noise,  interruptions  from  peers,  off-task  requests  from  the  boss  or 
family,  email,  and  an  array  of  similar  factors  due  to  the  “anywhere”  learning  environment 
that  e-  learning  presents.  Such  factors  “pull”  them  away  from  completing  the  course 
while  offsetting  factors  like  environmental  support  “push”  them  towards  completion 
(Catalano,  1985). 
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The  lessons  learned  from  the  retention  studies  should  still  apply  to  e-  learning. 

With  few  exceptions,  much  of  the  distance  education  research  suggests  that  similar 

outcomes  can  be  expected  from  students  that  participate  in  e-  learning  (Phipps  & 

Merisotis,  1999:1).  To  apply  lessons  from  classroom  settings,  one  must  first  consider 

whether  “push”  factors  like  environmental  support,  and  “pull”  factors  like  off-task 

demands  and  technical  problems  have  similar  effects  in  e-  learning  environments.  E- 

learning  course  designs  also  play  a  pivotal  role  in  determining  how  external  factors 

influence  completion  rates.  For  instance,  the  longer  the  course,  the  greater  the  chance  the 

e-leamer  could  become  distracted  and  focus  attention  elsewhere.  The  first  hypothesis 

predicts  the  likelihood  that  an  e-  learning  course  will  be  completed  given  the  amount  of 

technical  problems  and  off- task  demands  (pull  factors),  and  environmental  support  (push 

factors)  encountered  by  the  e-  learner. 

Hypothesis  la:  The  fewer  the  technological  problems,  the  more  likely  an 
individual  will  persist  at  e-  learning. 

Hypothesis  lb:  The  fewer  the  off-task  demands,  the  more  likely  an  individual 
will  persist  at  e-  learning. 

Hypothesis  Ic:  The  greater  the  environmental  support,  the  more  likely  an 
individual  will  persist  at  e-  learning. 

Understanding  that  external  factors  influence  e-  learning  completion  rates  is  not 
sufficient  for  e-  learning  course  designers  and  administrators  to  increase  the  likelihood 
that  students  will  persist  until  they  have  completely  finished  the  course.  Course  designers 
and  administrators  often  have  little  influence  on  technical  problems,  off  task  demands,  or 
environmental  support.  To  improve  e- learning  completion  rates,  practitioners  need  to 
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understand  how  external  factors  influence  human  motives  to  persist  at  e- learning. 
Research  in  this  area  may  produce  findings  that  help  e-  learning  scholars  and  practitioners 
design  motivationally  sound  courses  that,  in  turn,  improve  e-learning  completion  rates. 


Motivational  Constructs 


New  developments  in  technology,  performance  demands,  research,  and  a  host  of 
other  factors  influence  how  humans  view  the  world.  The  fundamental  ideas  behind 
human  behavior,  however,  have  remained  constant  (Lawler,  1994:xii).  For  fifty  years, 
human  motivation  has  been  described  as  the  energizing  force  that  prompts  people  to  act 
and  seek  out  particular  goals.  Motivation  represents  a  highly  complex  phenomenon  that 
affects,  and  is  affected  by,  a  multitude  of  internal  and  external  factors.  Some  well-known 
definitions  of  motivation  include: 

...[an  explanation  of]  how  behavior  gets  started,  is  energized,  is  sustained,  is  directed, 
is  stopped,  and  what  kind  of  subjective  reaction  is  present  in  the  organism  while  all 
this  is  going  on.  (Jones,  1955) 

...an  intra-  and  inter- individual  variability  in  behavior  not  due  solely  to  individual 
differences  in  ability  or  to  overwhelming  environmental  demands  that  coerce  or  force 
action.  (Vroom,  1964) 

...the  immediate  influences  on  the  direction,  vigor,  and  persistence  of  action. 
(Atkinson,  1964) 

...a  dynamic  resource  allocation  process  responsible  for  the  activation,  direction, 
intensity,  persistence,  and  termination  of  an  individual’s  behavior  holding  constant 
the  effects  of  personal  factors  (aptitude,  skills,  task  understanding,  etc.)  and 
environmental  constraints.  (Campbell  &  Pritchard,  1976:65) 

Motivational  Process 

Lawler  (1994)  argues  that  motivational  theory  serves  as  an  important  learning  tool 
because  it  presents  a  way  of  thinking  about  what  motivates  individuals  and  why  they  seek 
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to  accomplish  particular  goals.  Campbell  and  Pritchard  (1976)  identified  motivational 
constructs  as  forces  that  activate,  direct,  intensify,  persist,  and  terminate  behavior.  Steers 
and  Porter  (1991)  expanded  upon  this  concept  and  developed  the  motivational  process 
model  depicted  in  Figure  3. 


- ► 

Termination 

Need  fulfillment 
Rewards/Punishment 

Activation 

Need  deficiency 
Re-emergence  of  a  need 


intensity  &  Persistence 

Goal-directed  behavior/  Performance 


Direction 

Search  for  ways  to  satisfy  needs 


Figure  3:  Generalized  Model  of  the  Motivational  Process  (Steers  &  Porter,  1991) 


According  to  Steers  and  Porter,  “activation”  identifies  internal  and  external 
energetic  forces  that  drive  individuals  to  behave  in  certain  ways  (Steers  &  Porter, 

1991:6).  These  energetic  forces  are  normally  considered  in  the  context  of  human  needs, 
values,  or  goals.  “Direction”  identifies  the  decisions  a  person  makes  when  faced  with 
choosing  one  course  of  action  over  another  (Kanfer,  1990:79).  Direction  choice  is 
dependent  on  the  strength  of  the  need  discrepancy  or  the  relative  value  of  competing 
goals.  “Intensity”  identifies  the  amount  of  effort  given  towards  a  task  in  a  given  situation 
(Kanfer,  1990:79).  The  strength  of  the  effort  depends  on  two  factors:  1)  the  strength  of 
an  expectancy  that  the  behavior  will  lead  to  desired  goals;  and  2)  the  attractiveness,  or 
valence  of  the  goal  (Lawler,  1994:57).  “Persistence”  identifies  the  pattern  of  behaviors 
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over  time  (Kanfer,  1990:79).  Before  a  person  decides  to  persist,  they  take  a  systematic 
approach  by  considering  those  intrinsic  and  extrinsic  forces  that  either  reinforce  or 
dissuade  behavior  (Steers  &  Porter,  1991:6).  “Termination”  refers  to  those  factors  that 
cause  individuals  to  cease  or  redirect  behavior  (Kanfer,  1990:131).  Such  factors  can  take 
the  form  of  feedback,  self-evaluation,  and  goaf  attainment  (Steers  &  Porter,  1991:7). 

According  to  Lawler  (1994)  and  Steers  and  Porter  (1991),  any  discussion  of 
motivation  should  be  primarily  concerned  with  answering  questions  concerning  what 
activates  and  directs  behavior,  and  then  what  determines  the  intensity  of  the  goal  directed 
behavior  over  time.  Kanfer  (1990)  reviewed  a  variety  of  different  lines  of  motivational 
research  and  concluded  that  motivation  can  be  classified  as  either  distal  or  proximal. 
Distal  motivational  theories  primarily  emphasize  behavior  activation  and  direction  of 
goal  directed  behavior,  while  proximal  theories  explain  the  intensity  and  persistence  of 
goal  directed  behavior  over  time. 

Distal  Theories  of  Motivation 

Distal  theories  of  motivation  emphasize  processes  that  affect  goal  choice  and 
intended  future  effort.  They  are  key  to  understanding  the  effect  of  the  external  factors  on 
the  desire  to  take  an  e-  learning  course  (activation  and  direction).  Distal  theories  describe 
the  types  of  activities  that  people  choose  to  engage  in,  but  do  not  describe  how  people 
allocate  their  attention  once  they  are  performing  the  activity  (Kanfer,  1990).  Distal 
theories  help  with  the  assessment  of  potential  cost  versus  potential  benefit.  In  the  context 
of  completing  an  e- learning  course,  the  distal  system  of  motivation  would  explain  the 
priority  placed  on  completing  the  course  given  the  salience  of  other  activities  demanding 
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attention,  and  the  confidence  that  the  course  would  be  completed  given  the  amount  of 
available  resources. 

Proximal  Theories  of  Motivation 

Proximal  theories  of  motivation  emphasize  motivational  constructs  and  processes 
that  control  the  initiation  and  execution  of  actions  during  task  engagement  (intensity  and 
persistence).  They  are  also  key  to  understanding  the  effect  of  external  factors  and  the 
influence  of  course  design  on  e-learning  course  completion.  They  refer  to  discrete 
performance  accomplishments  aimed  towards  goal  attainment  (Kanfer,  1 990).  A  person 
operating  under  a  proximal  system  would  set  some  type  of  short-term  goal,  say  to 
complete  one  out  of  ten  e-  learning  modules  within  an  hour,  and  then  exert  energy  and 
effort  towards  accomplishing  that  goal,  with  subsequent  short-term  goals  to  follow. 

An  E-Learning  Example  of  Distal  and  Proximal  Theories  of  Motivation 

An  important  implication  of  the  distinction  between  the  two  types  of  theories  is 
that  the  same  variable  might  exert  a  different  effect  depending  on  whether  it  is  deployed 
within  the  distal  or  proximal  system  (Kanfer,  1990:82).  For  example,  in  distal  systems, 
individuals  that  are  computer  savvy  may  have  a  lot  of  self-confidence  and  ability  to 
successfully  complete  an  e-  learning  course.  In  the  proximal  system,  however,  this  high 
level  of  self-confidence  may  undermine  motivation  by  reducing  the  amount  of  time  and 
resources  this  person  allocates  towards  e- learning  (Kanfer,  1990:83). 

People  take  e- learning  courses  for  various  reasons.  These  reasons  include,  but  are 
not  limited  to:  increase  knowledge,  promotion  opportunity,  earn  credits  for  a  degree, 
pacify  a  need  for  accomplishment,  satisfy  curiosity,  or  some  combination  of  them  all. 
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Whatever  the  case  may  be,  students  view  e-learning  as  a  vessel  to  help  them  achieve 
some  desired  goal.  In  theory,  e-  learners  follow  a  chain  of  events  as  depicted  by  the 
motivational  process.  The  following  example  may  help  explain  the  motivational  process 
in  terms  of  an  individual  taking  an  e- learning  course  to  achieve  their  goal. 

Imagine  a  student  with  family  responsibilities,  who  would  like  to  receive  a 
promotion.  The  student’s  boss  will  only  promote  the  most  knowledgeable  employee.  To 
gain  more  knowledge,  the  student  can  attend  a  three- month  long  course  at  the  local 
university,  or  enroll  in  one  of  the  company’s  e-leaming  courses.  Attending  the 
university’s  course  will  require  the  student  to  spend  several  evenings  a  week  away  from 
the  family.  Spending  time  with  the  family,  however,  is  another  salient  desire  of  the 
student.  Company  policy  allows  the  student  to  work  on  e-leaming  courses  during 
business  hours,  which  frees  up  evenings  to  spend  with  family.  The  student’s  motivated 
behavior  to  gain  knowledge  is  activated  and  directed  towards  e-  learning  because  it 
possesses  the  most  attractive  and  timely  option  for  goal- attainment.  Distal  theories  of 
motivation  explain  the  student’s  choice  of  the  e-learning  course  over  a  traditional  course, 
or  no  course  at  all. 

After  enrolling  in  the  e-leaming  course,  the  student  finds  it  hard  to  engage  in 
course  work  because  of  competing  factors  like  job  demands,  office  noise,  intermptions, 
and  network  problems.  The  student  must  now  contend  with  the  decision  to  either  persist 
at  or  modify  (i.e.  re-direct  or  terminate)  the  goal-oriented  behavior.  If  the  “pushing” 
thoughts  of  having  more  money  for  their  family  are  enough  for  the  student  to  persist,  then 
there  is  a  good  chance  he  or  she  will  successfully  complete  the  course.  On  the  other 
hand,  if  the  “pulling”  forces  are  too  great  to  overcome,  the  student  may  decide  to  drop  the 
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e-learning  course  and  enroll  in  the  local  university’s  course  to  gain  the  desired 
knowledge.  Proximal  theories  of  motivation  explain  the  student’s  reactions  to  stimuli 
from  the  environment  and  help  explain  the  extent  that  attention  is  focused  on  the  e- 
learning  task.  Proximal  motivation  theory  can  also  explain  how  the  educational  goal  may 
change  over  time  leading  to  increased  attention  on  the  e-learning  and  eventual  course 
completion,  or  decreased  attention  ending  in  course  withdrawal  or  termination. 

Many  contemporary  theories  of  motivation  address  different  portions  of  the 
motivational  process.  They  attempt  to  explain  individual  behavior  in  specific  situations 
where  action  to  attain  desired  outcomes  are  prevalent.  A  review  of  these  theories,  as  well 
as  a  look  at  an  integrative  approach,  may  help  increase  our  understanding  of  those 
motivational  factors  that  influence  e-  learning  eourse  completion  rates. 

External  Effects  on  Activation  and  Direction  (Distal  Theories  of  Motivation) 

The  motivational  process  exerts  its  most  powerful  effects  on  e-  learning 
environments  by  influencing  the  individual’s  decision  to  work  on  the  task  rather  than  to 
do  something  else  (Kanfer,  1990:95).  Those  factors  that  activate  and  direct  individual 
motives  are  important  precursors  to  understanding  the  rationale  behind  intensity, 
persistence,  and  termination  of  goaf  directed  behavior.  Therefore,  this  section  reviews 
Needs-Motives- Value  and  Cognitive  Choice  approaches  to  motivation  in  an  attempt  to 
explain  why  people  are  activated  and  directed  to  take  e- learning  courses. 
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Need-Motive-Value  Approaches  to  Motivation 

Motivational  theories  that  focus  on  needs,  motives,  and  values  help  identify  those 
internal  and  external  forces  that  answer  the  question,  “What  activates  and  directs  human 
behavior?”  Need- motive- value  approaches  view  person-based  factors  as  major 
determinants  of  human  behavior  (Kanfer,  1990:81).  Early  need- motive- value  studies 
focused  on  personality  and  humanistic  needs  (Alderfer,  1969;  Maslow,  1943,  1954; 
McClelland  &  others,  1953).  These  studies  assumed  that  individuals  are  motivated  to  act, 
(or  in  the  case  of  this  study,  take  e-  learning  courses),  due  to  some  internal  tension  like 
hunger,  need  for  power,  or  need  for  Job  security  (Cherrington,  1991). 

Recent  needs- motive- value  studies  have  tended  to  focus  on  the  influence  of 
intrinsic  psychological  motives,  such  as  mastery,  control,  challenge,  competence,  and 
self-determination  (Deci,  1975;  Deci  &  Ryan,  1985;  Kanfer,  1990).  Deci  and  Ryan 
(1985)  defines  intrinsic  psychological  motives  as  innate,  needs  for  competence  and  self- 
determination.  Deci  and  Ryan  (1985)  believed  that  intrinsically  motivated  individuals 
seek  and  conquer  challenges  that  are  optimal  for  their  ability  and  would  take  e- learning 
courses  to  satisfy  such  things  as  a  need  for  mastery  (i.e.  competence),  or  a  need  for 
autonomy  (i.e.  self-determination). 

Similar  to  Deci  and  Ryan’s  theory,  deCharms  (1968)  viewed  self-determination, 
or  freedom  from  control,  as  the  necessary  and  driving  force  behind  intrinsic  motivation. 
According  to  deCharms,  a  person  with  a  high  need  for  locus  of  control  may  be  motivated 
by  the  fact  that  e- learning  allows  them  to  engage  in  learning  when,  where,  and  how 
desired  (deCharms,  1968:269). 
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Steers  and  Porter  (1991)  also  noted  that  any  complete  understanding  of  the  forces 

that  energize  human  behavior  must  take  into  account  the  nature  of  extrinsic  motivation. 

Extrinsic  motivation  refers  to  the  behavior  where  there  is  a  desire  to  do  something,  but 

not  out  of  interest  in  the  activity  itself  (Deci  &Ryan,  1985).  Extrinsically  motivated 

people  are  likely  to  initiate  behavior  when  it  is  perceived  that  salient  rewards  are  tied  to 

effective  performance  (Lawler,  1987).  Thus,  based  on  the  above  intrinsic- extrinsic 

discussion,  the  second  hypothesis  predicts  the  likelihood  that  an  e-leaming  course  will  be 

completed  given  intrinsic  and  extrinsic  motivational  factors. 

Hypothesis  2a:  The  greater  one’s  need  for  competence,  the  more  likely  they 
will  complete  the  e-  learning  course. 

Hypothesis  2b:  The  greater  one’s  need  for  self-determination,  the  more  likely 
they  will  complete  the  e-leaming  course. 

Hypothesis  2c:  The  greater  one’s  extrinsic  motivation,  the  more  likely  they 
will  complete  the  e-  learning  course. 

Findings  within  needs- motive -value  theories  have  led  researchers  to  recognize 
that  motives  play  an  important  role  in  activating  and  directing  human  behavior  (Kanfer, 
1990:1 12).  As  such,  this  research  contributes  to  the  understanding  of  the  theoretical 
formulations  that  predict  behavioral  choice  and  action  as  discussed  in  many  cognitive 
choice  theories  within  the  nomological  network  of  motivation. 

Cognitive  Choice  Approaches  to  Motivation 

Cognitive  choice  models  are  also  used  to  explain  activation  and  direction  of 
behavior  (Kanfer,  1990:82).  Expectancy  x  Valence  (E  x  V)  theories  dominate  the 
understanding  of  cognitive  processes  involved  in  the  decision-making  process.  These 
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theories  rest  on  the  assumption  that  people  will  seek  activities  that  increase  their  chances 
to  achieve  valued  outcomes  (Kanfer,  1990:1 13). 

Vroom  (1964)  perceived  expectations,  instrumentalities,  and  valence  as  the  key 
cognitive  variables  that  determine  motivational  force  and  choice.  Expectancies  were  seen 
by  Vroom  as  the  strength  of  a  person’s  belief  about  whether  a  particular  outcome  is 
possible  given  available  resources  (e.g.,  effort,  persistence,  time).  Vroom  defines 
instrumentalities  as  the  perceived  relationship  between  levels  of  performance  and  the 
probability  that  distal  outcomes  will  occur.  Valence  refers  to  the  anticipated 
attractiveness  of  each  distal  outcome  (Vroom,  1964). 

According  to  Vroom,  people  will  direct  resources  toward  a  particular  outcome 
when  they  have  a  high  confidence  that  the  can  achieve  the  outcome  given  resources  under 
their  control.  The  theory  suggests  that  people  make  relative  effort- to- reward  comparisons 
to  decide  on  the  most  promising  courses  of  action.  For  example,  individuals  may  enroll 
in  e-leaming  courses  because  they  expect  valuable  outcomes,  such  as  pay,  recognition,  or 
a  promotion  to  follow,  and  believe  that  spending  several  hours  per  day  engaged  with  e- 
learning  is  instrumental  to  attain  the  distal  outcome  of  course  completion. 

Similar  to  Vroom’s  E  x  V  theory,  Atkinson  (1957)  suggests  that  achievement 
motivation  is  a  multi-faceted  approach  consisting  of  four  elements:  motive,  probability 
(Pi),  incentive  (I,),  and  expectancy.  The  theory  implies  that  persons  differ  in  the  extent  to 
which  motives  for  success  (M,.)  are  stronger  than  motives  to  avoid  failure  (M/).  In  the 
context  of  e-leaming,  achievement  motivation  theory  suggests  that  individuals  with 
higher  motives  for  success  (M^  >  M/)  are  potentially  valued  students  because  they  plan 
ahead,  are  ready  to  be  confronted  with  difficulty,  are  independent  and  hardworking,  and 
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persist  longer  at  tasks  (Kanfer,  1990:1 15;  Ward,  1994:983).  Based  on  the  combination  of 
expectancy  theory  (Vroom,  1964)  with  achievement  motivation  (Atkinson,  1957),  the 
third  hypothesis  suggests  that: 

Hypothesis  3:  The  greater  an  individual’s  expectancy  for  success,  the  more  likely 

they  are  to  complete  an  e- learning  course. 

The  many  variables  of  cognitive  choice  theory,  in  one  form  or  another,  can  be 
related  to  goaf  directed  behavior  because  they  all  lead  to  some  desirable  endpoint. 
Therefore,  goals  could  possibly  have  a  direct  influence  on  whether  a  person  intensifies, 
sustains,  or  terminates  behavior  aimed  towards  course  completion.  Proximal  theories  of 
motivation  could  prove  beneficial  to  this  research  effort  because  they  investigate  goals 
and  the  role  they  play  in  motivating  human  behavior. 

External  Effects  on  Intensity  and  Persistence  (Proximal  Theories  of  Motivation) 

Proximal  theories  of  motivation  focus  on  the  self-governing  concepts  of  the 
motivational  processes  underlying  goal  attainment  (Kanfer,  1990:131).  More  precisely, 
proximal  motivational  theorists  suggest  that  goals  represent  a  critical  component  of  the 
self- regulation  process  because  they  provide  the  mechanism  by  which  motivational  states 
are  translated  into  action  or  terminated  (Kanfer,  1970;  Kanfer,  1990;  Kanfer  &  Schefft, 
1988;  Latham  &  Locke,  1979;  Locke,  1967).  Therefore,  this  section  reviews  Goal- 
Setting  and  Self-Regulation  research  in  an  attempt  to  identify  those  factors  that  influence 
the  intensity  and  persistence  of  goal-oriented  behavior. 
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Goal  Approaches  to  Motivation 

Locke’s  (1968)  goal-setting  theory  proposed  that  goals  are  the  driving  force 
behind  mobilizing  on- task  effort  and  encouraging  task  persistence  (Locke,  1968;  Locke 
&  others,  1981).  Intensity  is  one  of  the  major  goaf  setting  attributes  in  Locke’s  theory. 
Intensity  refers  to  the  strength  of  the  goal  in  relation  to  perceived  goal  importance  and 
amount  of  goal  commitment  obtained  (Locke,  1968).  Goals  that  are  perceived  as  “very 
important”  are  more  likely  to  have  stronger  goal  intensity  than  those  perceived  as 
“unimportanf ’,  or  are  less  valuable.  For  instance.  Defense  Acquisition  personnel  must 
complete  e-learning  courses  to  obtain  job  certification.  Locke’s  theory  posits  that  the 
goal  intensity  to  complete  these  courses  is  stronger  for  Defense  Acquisition  personnel 
than  for  those  taking  the  courses  out  of  curiosity. 

Steers  and  Porter  make  a  similar  argument  and  suggest  that  certain  environmental 
forces  can  influence  goal  importance.  An  e-  learner  who  is  not  required  to  complete  a 
course  until  the  following  week,  may  decide  to  expend  effort  towards  some  off-task 
(Steers  &  Porter,  1991).  But,  if  that  e- learner’s  boss  expects  the  course  to  be  completed 
by  day’s  end,  the  e- learner  will  likely  intensify  efforts  towards  completing  the  course  due 
to  fear  of  not  meeting  the  boss’s  expectations.  In  this  case,  the  boss’  expectations 
increased  the  goal  intensity  of  completing  the  e- learning  course.  Given  these  arguments 
on  goal  intensity,  it  is  hypothesized  that: 

Hypothesis  4:  The  greater  the  goal  intensity,  the  more  likely  the  e- learner  will 

complete  the  e-learning  course. 

Raynor  and  Roeder  (1987)  expanded  Atkinson’s  original  theory  of  achievement 
motivation  and  suggested  that  many  goals,  such  as  career  advancement,  require 
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performance  of  a  hierarchical  series  of  tasks.  According  to  Raynor  and  Roeder’s  theory, 
a  person  will  divide  a  larger  goal  (e.g.,  get  a  college  degree),  into  several  smaller  tasks  or 
goals  (e.g.,  pass  Chemistry  101).  Each  new  task  within  the  series  is  contingent  on 
successful  completion  of  the  prior  tasks  (Raynor  &  Roeder,  1987).  Raynor  and  Roeder 
(1987)  theorized  that  as  the  number  of  steps  in  the  contingent  path  increases, 
achievement- oriented  persons  (M  v  >  M/)  should  demonstrate  higher  levels  of  motivation 
than  failure- oriented  persons  (M/>  M,)  on  the  first  step.  In  addition,  the  anticipated  time 
it  takes  to  complete  the  goal  would  increase.  According  to  their  theory,  this  increase  in 
time  it  takes  to  achieve  a  goal  decreases  the  beneficial  effects  of  contingent  paths  on 
motivation  (Raynor  &  Roeder,  1987). 

Raynor  and  Roeder’s  (1987)  findings  suggest  that  course  length  (i.e.  average  time 

it  takes  to  complete  a  module  and  time  given  to  complete  course)  could  prove 

instrumental  in  determining  the  amounts  of  effort  and  persistence  put  forth  towards  the 

course.  For  instance,  failure-oriented  students  may  feel  threatened  by  e- learning  if  the 

course  is  perceived  as  being  too  long.  In  this  case,  the  amount  of  effort  and  persistence 

put  forth  towards  the  course  will  likely  decrease.  Conversely,  intensity  and  persistence 

may  increase  for  shorter  courses  because  they  are  seen  as  “low  hanging  fhih”  in  which 

goal- attainment  is  perceived  as  quick  and  easy.  The  fifth  hypothesis  predicts  the 

likelihood  that  an  e- learning  course  will  be  completed  given  the  length  of  the  e- learning 

course  and  achievement  motives  of  the  e-  learner. 

Hypotheses  5:  E- learning  course  length  in  terms  of  the  time  it  takes  to  complete  a 
module  and  the  total  time  given  to  complete  the  course,  will  have  different  effects 
on  completion  rates  for  success  and  failure  oriented  students. 
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Considerable  differences  can  exist  among  individuals  concerning  the  manner  and 
intensity  in  which  they  select  certain  motives  over  others  (Steers  &  Porter,  1991:7).  Self¬ 
regulation  theorists  suggest  that  self-governing  factors  such  as  self-efficacy,  feedback, 
and  historical  events  and  experiences,  whether  satisfactory  or  dissatisfactory,  helps  de¬ 
conflict  goals  because  of  the  mediating  affect  they  have  on  the  executive  processes  that 
guide  goal  choice  and  effort  (Kanfer,  1990:124). 

Self-regulation  Approaches  to  Motivation 

The  concept  of  self-efficacy  receives  much  attention  in  self- regulation  research. 
Bandura  defines  self-efficacy  as  “the  historical  judgment  an  individual  makes  about  his 
or  her  ability  to  execute  a  particular  behavior”  (Bandura,  1978:240).  Wood  and  Bandura 
(1989)  expanded  upon  this  definition  by  suggesting  that  self-efficacy  judgments  form  a 
central  role  in  the  regulatory  process  because  they  determine  how  much  effort  people  will 
spend  on  a  task  and  how  long  they  will  persist  with  it.  Self-efficacy  theory  suggests  that 
there  are  four  major  sources  of  information  used  when  forming  self-efficacy  judgments 
(Bandura,  1977).  Figure  4  identifies  these  sources. 

Personal  accomplishments  refer  to  past  experiences  with  the  specific  task  being 
judged.  Vicarious  experience,  also  referred  to  as  modeling,  is  gained  by  observing  others 
perform  activities  successfully.  Social  persuasion  refers  to  activities,  such  as  coaching 
and  feedback,  where  people  are  led  to  believe  that  they  can  successfully  accomplish  the 
task  (Bandura,  1977;  Bandura  &  Cervone,  1986).  Physiological  and  emotional  states 
refer  to  an  individual’s  state  of  mind  with  respect  to  a  specific  task  (Bandura,  1988). 
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Figure  4:  Self-EfiRcacy  Theory  (Bandura,  1977) 


Self-efficacy  theory  has  been  used  in  numerous  studies,  including  those  of 
information  technology  (Compeau,  1992;  Hill,  Smith,  &  Mann,  1987;  Staples,  Hulland, 

&  Higgins,  1998)  and  learning  domains  (Gist,  1989).  In  particular.  Staples,  Hulland,  and 
Higgins  (1998)  concluded  that  if  organizations  can  learn  how  to  increase  their 
employees’  self-efficacy  judgments  about  their  abilities  to  complete  relevant  tasks,  this 
should  lead  to  improved  performance. 

These  results  suggest  e- learners  with  positive  beliefs  about  their  ability  to  perform 
tasks  will  exert  greater  efforts  towards  completing  an  e-  learning  course,  while  those  with 
negative  self-beliefs  are  likely  to  reduce  their  efforts,  switch  to  an  off-task,  or  quit 
altogether  (Bandura  &  Schunk,  1981;  Weinberg,  Gould,  &  Jackson,  1979).  The  sixth 
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hypothesis  predicts  the  likelihood  that  an  e- learning  course  will  be  completed  given  the 
level  of  an  e- learner’s  self-efficacy. 

Hypothesis  6:  The  greater  one’s  self-efficacy  in  e- learning  environments,  the 
more  likely  they  will  complete  the  e-  learning  course. 

Bandura  (1982)  and  Kanfer  (1970)  take  a  different  look  at  self- regulation 
processes.  They  view  self- regulation  as  comprised  of  three  interrelated  components — 
self- observation,  self-evaluation,  and  self-reaction.  Self-observation  refers  to  the 
knowledge  gained  from  feedback  about  those  features  of  activity  most  significant  for  goal 
attainment  (Bandura,  1982,  1986).  Self-evaluation  allows  a  person  to  compare  their 
desired  goal  state  with  their  observed  performance.  Self-reactions  are  the  satisfactory  or 
dissatisfactory  internal  responses  that  occur  in  response  to  self-evaluation.  Goal- 
performance  discrepancies  occur  when  performance  is  observed  as  falling  short  of 
perspeetive  goals. 

Feedback  moderates  the  relationship  between  goals  and  performance.  Feedback 

comes  into  play  in  one’s  subsequent  effort  to  achieve  desired  results  (Steers  &  Porter, 

1990,  186).  Studies  show  that  when  workers  got  feedback  relevant  to  the  specific  goal, 

performance  improved  (Steers  &  Porter,  1990:360).  These  results  lead  one  to  infer  that 

feedback  information  is  important  because  it  provides  individuals  with  a  means  to  assess 

their  performance.  Therefore,  based  on  the  research  by  Bandura  (1982)  and  Kanfer 

(1970),  it  appears  reasonable  to  hypothesize  the  following: 

Hypothesis  7a:  E-leamers  that  believed  they  received  sufficient  and  timely 
feedback  information  are  more  likely  to  complete  e-  learning  courses. 

Hypothesis  7b:  E-learning  courses  that  provide  feedback  information  that  shows 
progress  towards  proximal  and  distal  goals  will  have  higher  completion  rates  than 
courses  that  do  not. 
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Wiener’s  (1971)  attribution  theory  suggests  that  individuals  make  causal 
explanations  about  past  behavior  that  determine  future  levels  of  intensity  and  persistence. 
These  causal  explanations  can  be  explained  by  internal  (e.g.,  effort  and  ability)  and 
external  (e.g.,  task  difficulty  and  luck)  forces.  Figure  5  summarizes  these  attributional 
forces  Ifom  an  e-learning  perspective. 


Internal 

External 

Ability 

Task  difficulty 

Stable 

Cognition  level 

Poor  course  design 

Unstable 

Effort 

Luck 

Time  put  towards 
course 

Internet  problems 

Figure  5:  Attribution  Classification  Scheme  (Wiener,  1972) 

The  primary  attribute  of  concern  for  e- learning  practitioners  is  effort.  E- learning 
designers,  instructors,  and  administrators  must  keep  people  willing  to  put  forth  effort 
even  if  the  other  three  attributes  are  poor  or  missing.  Findings  within  attribution  theory 
suggest  that  changes  in  an  individual’s  expectancy  for  goal  attainment  are  primarily 
influenced  by  perceived  stability  of  outcome  causes. 

According  to  Wiener’s  theory,  the  perception  that  effort  can  be  increased 
stabilizes  or  enhances  expectations  for  future  success  with  the  e-leaming  course  (Wiener, 
1986).  In  addition,  the  ability  attribute  is  a  stable,  internal  motivational  factor  that  is 
under  volitional  control  and  can  therefore  be  modified  to  suffice  the  goal- oriented 
situatbn  (Weiner,  1972).  Conversely,  an  e- learner  that  repeatedly  fails  at  e-leaming,  may 
eventually  attribute  the  failure  to  Internet  problems,  which  are  considered  uncontrollable. 
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external  factors  (Wiener,  1986).  Therefore,  this  e- learner  will  likely  expect  failure  to 
continue  and  terminate  behavior  (Wiener,  1986).  The  eighth  hypothesis  predicts  the 
likelihood  that  an  e-  learning  course  will  be  completed  given  the  amount  of  external 
difficulty  encountered  in  terms  of  course  design. 

Hypothesis  8:  The  greater  the  amount  of  external  difficulty  encountered  in  terms 

of  course  design,  the  more  likely  the  e-  learner  will  drop  out  of  the  course. 

Theories  in  self- regulation  provide  a  framework  for  understanding  cognitive 
determinants  of  goal-directed  behavior  (Kanfer,  1990:133).  They  also  allow  individuals 
to  interpret  performance  and  judge  themselves  in  relation  to  goal  driven  behaviors. 
Similar  to  the  understanding  of  external  factors,  individual  theories  of  motivation  are  not 
sufficient  to  determine  why  people  fail  to  complete  e- learning  courses  (Kanfer,  1990:81). 
An  approach  that  integrates  several  motivational  theories  into  one  may  prove  more 
appropriate  to  explain  the  complex  nomological  network  of  the  motivational  process. 

An  Integrative  Approach 

Kanfer  and  Ackerman  (1989)  introduced  the  Integrative  Resource  Model  of 
Ability-Motivation  Interactions  for  Attentional  Effort  as  an  approach  to  describe  an 
individual’s  performance  based  on  allocation  of  resources  towards  a  task.  Analyzing 
tasks  in  this  manner  allows  for  the  discovery  of  differences  between  tasks  according  to 
the  affects  attentional  demands  have  on  behavior  and  performance  (Kanfer  &  Ackerman, 
1989).  For  purposes  of  this  study,  the  task  refers  to  e- learning  while  performance  is 
measured  in  terms  of  whether  the  course  was  completed  or  not. 


35 


Kanfer  and  Ackennan  posit  that  the  mapping  of  motivational  processes  to  task 
performance  involves  the  simultaneous  operation  of  three  cognitive  mechanisms  called 
performance-utility,  effort-utility,  and  perceived  effort-performance  (Kanfer  & 
Ackerman,  1989).  The  performance-utility  function  refers  to  the  value  placed  on  any 
perceived  benefits  associated  with  different  levels  of  performance.  The  effort- utility 
function  refers  to  the  perceived  costs  and  benefits  of  expending  effort  towards  a 
particular  goal.  The  perceived  effort- performance  function  brings  together  the  two 
functions  of  performance-utility  and  effort-utility  and  allows  a  person  to  judge  the  point 
at  which  costs  of  effort  override  the  benefits  (Kanfer,  1990:148). 

Similar  to  Expectancy  x  Value  theories,  these  three  mechanisms  hypothesize  that 
an  individual’s  expectancy  that  a  given  action  will  be  followed  by  a  given  result,  and  the 
amount  of  effort  put  towards  that  result,  determines  the  motivational  force  expended 
(Vroom,  1964).  In  addition,  the  effort- utility  function  mirrors  the  portion  of  Catalano’s 
motivation-retention  model  that  highlights  the  cost  and  benefits  associated  with  effort 
applied  in  terms  of  attention  and  energy  (Catalano,  1985). 

Kanfer  and  Ackerman’s  integrative  model,  shown  in  Figure  6,  purports  that  the 
amount  of  resources  used  towards  a  task,  and  how  those  resources  are  allocated,  depend 
on  motivational  processes  (Kanfer  &  Ackerman,  1989).  According  to  their  theory,  both 
distal  and  proximal  processes,  as  well  as  feedback,  affect  the  manner  in  which  resources 
are  allocated  to  off-task,  task,  and  self-regulation.  Distal  and  proximal  processes  are  in 
turn  affected  by  goal  setting.  Self- regulatory  processes  help  determine  changes  in  the 
allocation  policy,  as  well  as  changes  in  the  perceived  effort-performance  function. 
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Figure  6:  Integrative  Resource  Model  of  Ability-Motivation  Interactions  for  Attentional 

Effort  (Kanfer  &  Ackerman,  1989) 

In  addition,  the  Kanfer  and  Ackerman  model  assumes  that  when  tasks  are  difficult,  or 
task  demands  are  high,  the  individual  allocates  more  effort  and  persistence  towards  the 
task  (Kanfer  &  Ackerman,  1989). 

The  model  has  demonstrated  successful  results  under  field  conditions  in  previous 
Air  Force  studies  involving  computer  systems  and  learning  (Kanfer  &  Ackerman,  1989). 
Therefore,  it  may  prove  helpful  in  determining  the  motivational  impact  levied  on 
performance  in  e-learning  environments  while  providing  a  heuristic  view  of  how 
motivation  influences  e-learning  course  completion  rates.  For  instance,  an  integrative 
approach  may  help  determine  how  e-  learners  that  engage  with  e-  learning  in  noisy 
environments  allocate  their  attentional  resources.  Efforts  may  help  alleviate  such 
distractions  so  that  individuals  can  better  allocate  the  necessary  attention  towards  the  goal 
at  hand:  e-  learning. 
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As  can  be  seen  from  Kanfer  and  Ackennan’s  integrative  study,  motivation  is  a 
complex  phenomenon  that  can  be  best  understood  within  a  multivariate  framework 
(Kanfer,  1990;  Steers  &  Porter,  1991).  Some  theorists  argue  that  researchers, 
practitioners,  and  scholars  must  take  such  complex  factors  into  consideration  if  they  are 
to  properly  evaluate  the  adequacy  of  motivational  theory  in  explaining  behavior  in 
learning  environments  (Steers  &  Porter,  1991:23). 

E-Learning  and  Motivation:  Some  Concluding  Observations 

Some  theorists  believe  that  the  ultimate  question  of  motivation  comes  down  to  the 
complex  interaction  between  the  “push”  forces  within  persons  and  “pull”  forces 
originating  from  the  environment  (Steers  &  Porter,  1991:108).  Recall  push  forces  refer 
to  the  positive  mechanisms  that  drive  a  person  towards  goals  or  desires,  while  the  pull 
forces  are  the  negative,  restraining  cues  that  thwart  or  offset  goal- oriented  behavior. 
General  findings  from  this  research  would  suggest  that  persistence,  or  lack  thereof,  is  a 
chief  determinant  of  whether  a  student  stays  or  leaves  the  learning  environment. 

Other  findings  support  the  notion  that  different  forms  of  motivation  may  come 
about  more  through  conjunction  of  specific  person-environment  matches  (Lepper,  1985; 
Lepper  and  Chabay,  1985;  Lepper  and  Malone,  1987).  Such  findings  suggest  that  people 
may  be  stimulated  by  different  instructional  methods  that  are  more  “in-  line”  with  their 
goals  and  desires.  For  instance,  computerized  training  has  raised  questions  as  to  whether 
individuals  differ  in  their  responsiveness  to  instructional  environments  containing 
specific  motivational  embellishments  (Kanfer,  1990:96).  If  as  self- regulation  theory 
argues,  goals  are  the  most  potent  determinant  of  action,  and  scholars  believe  that  most 


38 


individuals  enroll  in  e-  learning  courses  with  the  goal  of  completing  them  in  mind,  then 
researchers  must  determine  at  what  point  do  people  stop  putting  forth  effort,  and  why. 

Therefore,  the  objective  of  this  study  is  to  discover  those  motivational  factors  that 
influence  people  to  stay  in  or  drop  out  of  an  e- learning  environment.  As  can  be  seen  with 
the  complex  nomological  network  of  the  motivational  process,  an  analysis  of  motives 
may  be  best  viewed  using  an  integrative  approach.  An  attempt  to  capture  an  integrated 
motivational  approach,  similar  to  Kanfer  and  Ackerman’s  (1989),  for  this  study  is 
illustrated  in  Figure  7. 


Figure  7:  Integrated  Motivational  Approach  to  E-Learning  Course  Completion  Model 


The  model  tries  to  capture  the  impact  that  external  factors  have  on  motivational 
factors.  Motivational  factors  are  in  turn  related  to  the  complex  nomological  network  of 
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motivated  behavior  in  terms  of  completing  an  e-leaming  course.  As  discussed  earlier, 
goals  and  desires  normally  activate  and  direct  behavior.  The  intensity  of  goal- directed 
effort  is  then  determined  by  perceived  outcomes  and  goal  valence.  From  there,  a 
person’s  expectance  of  goal  attainment  determines  how  long  they  are  willing  to  persist  at 
e-leaming  in  order  to  complete  the  course. 

Task  persistence  is  further  influenced  by  the  person’s  self-efficacy  in  his  or  her 
ability  to  successfully  complete  the  course  work.  Progress  towards  goal-attainment  is 
assessed  by  self-regulation  concepts  and  feedback.  Self-regulation  and  feedback 
information  are  then  compared  to  the  perceived  outcomes  and  expectancies  that  initially 
instigated  the  direction,  intensity,  and  persistence  of  goal- directed  behavior.  The  result  is 
one  of  two  things:  1)  continued  effort  and  task  persistence  (which  leads  to  course 
completion),  or  2)  terminated  behavior  (which  results  in  the  person  dropping  out).  The 
external  factors  of  technical  problems,  off-task  demands,  and  environmental  support, 
along  with  the  course’s  design,  influence  motivated  behavior  throughout  the  entire 
motivational  process.  The  hypotheses  introduced  throughout  the  literature  review  are 
incorporated  within  the  model  as  a  means  to  show  their  relation  to  the  different  constmcts 
and  processes  that  influence  behavior. 

Research  findings  from  such  models  may  provide  educators  and  trainers  insights 
on  incorporating  motivational  features  into  e-  learning  programs  that  significantly 
improve  attrition  rates.  Additionally,  future  studies  may  be  able  to  use  these  findings  to 
develop  an  integrative  motivational  model  that  specifically  aims  to  predict  and  explain 
course  completion  rates  within  e-  learning  environments. 
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III.  Methodology 


“The  process  of  data  analysis  is  eclectic;  there  is  no  right  way.  ” 

Renata  Tesch  (1990) 


Introduction 

This  chapter  describes  the  methodology  used  to  identify  and  measure  those 
motivational  factors  that  promote  or  inhibit  e-  learning  course  completion  rates.  A 
literature  review,  as  well  as  interviews,  content  analysis,  and  observations,  was  used  to 
develop  the  research  instrument  used  in  the  study.  The  research  instrument,  entitled  the 
E-Leaming  Course  Questionnaire  (ECQ),  was  then  employed  to  elicit  and  extract 
information  to  address  the  research  questions  posed  in  Chapter  I  and  test  the  hypotheses 
introduced  in  Chapter  II.  Eight  (8)  e-leaming  courses — five  from  the  Air  Force  Institute 
of  Technology  Virtual  Schoolhouse  and  three  from  the  Defense  Acquisition  University 
Virtual  Campus  (DAU) — were  targeted  for  analysis.  The  ensuing  sections  describe  the 
research  approach,  course  characteristics,  instrument  development,  subject  pool,  data 
collection  procedures,  and  statistical  analysis  used  in  the  research  effort. 

Research  Approach 

The  method  used  to  gather  information  for  the  investigation  was  theor}^  driven 
using  the  ECQ  to  collect  quantitative  data.  An  overview  of  all  the  steps  used  in  the 
research  follows: 
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1.  conduct  interviews  with  subject  matter  experts, 

2.  observe  e- learning  as  an  unobtrusive  participant, 

3.  analyze  previous  e-learning  course  critiques, 

4.  review  e-learning-  and  motivation-related  literature, 

5.  select  e-leaming  courses  (referred  to  as  the  “target”  courses), 

6.  analyze  the  characteristics  of  the  selected  courses, 

7.  develop  the  research  instrument, 

8.  select  subjects  from  targeted  courses  populations, 

9.  administer  the  survey, 

10.  gather  survey  results, 

1 1 .  perform  statistical  analysis  of  the  final  data,  and 

12.  interpret  the  results. 

Interviews 

Multiple  interviews  were  conducted  with  nine  subject  matter  experts  between  the 
dates  of  30  January  2001  and  10  June  2001.  Each  interview  lasted  approximately  one 
hour.  The  subject  matter  experts  included  personnel  from  AFIT,  DAU,  and  Northrop 
Grumman  Corporation  (company  that  designed  and  built  the  Virtual  Schoolhouse  e- 
leaming  courses).  Interview  questions  were  designed  to  extract  information  on 
perceptions  of  those  factors  that  influenced  e-  learning  course  completion  rates  the  most. 
Though  an  interview  guide  was  used,  the  actual  questions  asked  during  interview 
sessions  were  composed  on  the  spot.  This  interview  approach  was  taken  because  it 
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helped  fit  the  questions  to  the  natural  rhythm  of  the  dialogue  as  well  as  promote 
maximum,  unbiased  disclosure  by  the  interviewee  (Dooley,  2001:258). 

Observation 

The  researcher  received  permission  to  gain  entry  into  the  VSH  Operational 
Safety,  Suitability,  and  Effectiveness  (OSS&E)  course  and  participated  as  an  unobtrusive 
observer  from  23  April  2001  (course  start  date)  until  19  May  2001  (course  end  date). 

The  advantages  that  observation  brings  to  research  include:  1)  researcher  gains  firsthand 
experience  with  environment  and  subjects,  2)  researcher  can  record  information  as  it 
occurs,  and  3)  unusual  aspects  can  be  captured  during  observation  (Creswell,  1994:150; 
Dooley,  2001:255). 

Content  Analysis 

Once  the  observation  phase  ended,  end-of-course  critiques  were  reviewed  and 
analyzed  from  six  of  the  target  courses.  The  information  extracted  from  the  critiques  was 
compared  to  both  the  interview  and  observation  data  collected  in  an  attempt  to  identify 
and  compare  similarities,  accuracies,  and  frequency  of  responses.  The  objective  here  was 
to  identify  recurring  e-  learning  trends  that  influence  course  completion  rates. 

Quantitative  Design  Objective 

One  advantage  of  employing  a  quantitative  design  is  that  theory  and  literature  are 

used  deductively  to  help  guide  the  study  toward  answering  the  research  questions 

(Creswell,  1994:179).  The  information  collected  through  quantitative  procedures  can  be 

analyzed  statistically  to  generalize  from  the  data  and  support  or  refute  theories.  The 

objective  of  this  design  is  to  extend  motivational  knowledge  while  identifying  e- learning 
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course  designs  that,  in  conjunction  with  motivational  constructs,  identify  differences 
between  those  that  complete  e-  learning  courses  and  those  that  do  not.  Therefore,  the 
motivational  constructs — activation,  direction,  intensity,  persistence,  and  termination — 
discussed  in  the  literature  review  of  Chapter  II  are  analyzed  in  relation  to  e- learning 
course  designs.  The  characteristics  of  course  length,  course  difficulty,  amount  of 
feedback  provided,  and  whether  or  not  the  course  was  a  “requirement,”  are  collectively 
used  by  this  research  to  define  e-  learning  course  design.  Course  characteristics  are 
compared  to  both  ECQ  data  and  course  completion  rates  to  determine  if  there  are  any 
significant  relationships  between  the  three. 

Course  completion  rates  refer  to  the  percentage  of  students  that  actually  complete 
the  e- learning  course  compared  to  the  number  initially  enrolled.  The  following  equation 
was  used  to  calculate  the  completion  rates  for  the  eight  courses  used  in  this  study: 

E'er  =  number  initially  enrolled  -  number  that  dropped  (1) 

number  enrolled 

Number  that  dropped  corresponds  to  those  that  did  not  complete  the  course.  For 
purposes  of  this  research,  the  term  dropped  is  defined  as  “the  failure  to  complete  a  given 
course  of  action  or  attain  a  desired  goal  for  which  he  or  she  first  entered”  (Tinto,  1982). 
Students  that  end  the  course  with  a  status  of  withdrawn,  failed,  or  incomplete  will  be 
considered  as  dropped  from  the  course.  Information  as  to  the  number  of  students  initially 
enrolled  and  the  number  of  students  that  dropped  was  collected  from  the  Virtual 
Schoolhouse  database,  or  provided  by  Defense  Acquisition  University  Virtual  Campus 
course  administrators. 
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Target  Course  Selection 

The  eight  target  courses  used  in  the  study  provide  information,  instruction,  and 
continuing  education  related  to  systems  acquisition  management.  With  each  course, 
certain  amounts  of  points  or  units  are  earned  for  course  completion.  Some  of  these 
courses  are  required  to  be  taken  while  others  are  not.  Defense  Acquisition  University 
Virtual  Campus  (DAU)  courses  provide  mandatory,  assignment- specific,  and  continuing 
education  courses  for  military  and  civilian  acquisition  personnel.  Their  mission  is  to 
provide  the  acquisition  community  with  the  right  learning  products  and  services  to  make 
smart  business  decisions  (DoD  Directive  5000.57,  1991).  Systems  acquisition  career 
fields  must  take  and  complete  certain  DAU  courses  to  become  certified  in  their  specialty. 
They  must  then  take  and  complete  follow-on  e-learning  courses  to  maintain  this 
certification.  Therefore,  as  motivational  theory  would  posit,  career-related  motives  may 
be  the  activating  (i.e.  course  is  required)  and  intensifying  (i.e.  for  certification)  force 
behind  course  completion  for  the  DAU  courses  (Locke,  1968).  Virtual  Schoolhouse 
courses  are  not  required  to  be  taken,  however,  Defense  Acquisition  policy  states  that 
certified  acquisition  professionals  shall  earn  a  minimum  of  80  Continuous  Learning 
Points  (CLP)  every  two  years  to  stay  current  in  the  profession  (Defense  Acquisition 
Workers  Improvement  Act,  1993). 

Course  completion  rate  {£„)  was  the  primary  factor  used  to  select  the  eight  target 
courses.  A  cumulative  £„  was  calculated  for  all  Virtual  Schoolhouse  courses  in  session 
between  the  months  of  May  through  December  2001.  Cumulative  rates  were  used 
because  each  Virtual  Schoolhouse  course  was  in  session  more  than  once  during  this  time 
frame.  Each  Virtual  Schoolhouse  target  course  was  then  selected  based  on:  1)  its  E^,  and 
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2)  its  course  characteristics.  The  courses  with  the  highest  and  lowest  Ecr  became 
automatic  targets.  The  other  three  Virtual  Schoolhouse  target  courses  were  chosen  based 
on  course  length,  initial  number  of  enrollees,  and  course  description.  This  helped 
generalize  the  study  as  well  as  ensure  that  several  E^r  perspectives  (i.e.  high,  medium,  and 
low)  were  analyzed. 

The  Defense  Acquisition  University- Wright- Patterson  Division  Associate  Dean  of 
Academic  Affairs,  Mr.  Travis  Stewart,  chose  the  three  DAU  target  courses  used  in  the 
study.  He  chose  one  course  with  a  high  Ecx,  one  course  with  a  low  E^,  and  one 
moderately  difficult  course,  in  terms  of  comprehension  and  understanding,  to  obtain  a 
good  cross-section  of  DAU  courses.  A  cumulative  E^  for  the  period  between  May  and 
December  200 1  was  also  calculated  for  the  DAU  courses  chosen  for  this  study. 

To  objectively  collect  data  on  motivational  factors  that  influence  completion 
rates,  the  ECQ  was  administered  to  randomly  selected  subjects  previously  enrolled  in  the 
target  courses.  The  questionnaire’s  intent  was  to  assess  motivational  factors  that  answer 
questions  as  to  when,  where,  and  how  a  student’s  desire  to  complete  the  e-learning  course 
is  positively  or  negatively  influenced.  Therefore,  ECQ  responses  were  analyzed  along 
with  target  course  characteristics  in  an  attempt  to  determine  which  prevailing 
motivational  factors  caused  people  to  complete  or  drop  out  of  e- learning  courses. 

Course  Characteristics 

Target  course  names,  numbers,  and  descriptions,  along  with  their  course 
completion  rates  {Ecr)  and  whether  they  were  required  or  not,  are  provided  in  Table  1. 
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Course  Length 

This  study  looks  at  and  compares  two  aspects  of  course  length — average  time 
required  to  complete  a  module  and  the  amount  of  time  given  to  complete  the  entire 
course.  The  amount  of  time  given  to  complete  a  course  is  directly  affected  by  the 
course’s  subject  material,  lesson  plan,  and  objective.  For  Virtual  Schoolhouse  courses, 
the  averaged  time  (in  hours)  required  to  complete  a  module  is  derived  by  dividing  CLP 
by  the  number  of  modules  in  the  same  course.  For  example,  if  a  particular  Virtual 
Schoolhouse  eourse  is  worth  10  CLPs  and  has  20  modules,  then  it  would  be  expected 
that,  on  average,  it  would  take  one- half  (0.50)  hour  to  complete  each  module.  DAU  uses 
the  same  ealculation  for  their  average  time  to  complete  a  module,  however,  they  use 
Continuing  Education  Units  (CEU)  that  are  depicted  in  tenths  (i.e.  10  CEU  =1.0  hour). 
See  Table  2  for  a  breakdown  of  target  course  length  characteristics. 


Table  2:  Distinguishing  Characteristics  of  Target  Course  Length 


Course  Name 

Average 
Time  per 
Module 
(in  hours) 

Time 
Given  to 
Complete 
(in  days) 

Number 

of 

Modules 

CLP 

or 

CEU 

Current  Topics  in  Financial  Management 

1.60 

42 

10 

16 

Weapons  Systems  Pollution  Prevention 

2.86 

42 

14 

40 

Advanced  Concept  Technology  Demonstrations 

1.86 

28 

8 

15 

Integrated  Product  Support 

2.14 

42 

14 

30 

Modification  Management 

.75 

28 

8 

6 

Acquisition  Logistics  Fundamentals 

1.50 

60 

16 

2.4 

Fundamentals  of  Systems  Acquisition  Management 

1.00 

60 

24 

2.5 

Reliability  and  Maintainability 

2.16 

60 

7 

1.7 
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Course  Difficulty 

For  purposes  of  this  research,  course  difficulty  refers  to  how  hard  e- learning 
courses  are  in  terms  of  their  competence  level,  readability,  and  whether  the  course 
requires  100-percent  mastery  or  not.  The  competency  level  of  each  target  course  is  based 
on  Bloom’s  Taxonomy  of  Educational  Objectives  (Bloom,  1956)  (Appendix  A).  Bloom 
categorized  competency  in  terms  of  the  level  of  abstraction  required  to  answer  commonly 
asked  questions  in  educational  settings.  Each  target  course  is  numerically  rated  based  on 
Bloom’s  Taxonomy,  with  “1”  =  Knowledge  Level,  and  “2”  =  Comprehension  Level. 
Though  Bloom’s  Taxonomy  goes  up  to  the  Evaluation  Level,  comprehension  is  the 
highest  level  of  understanding  of  any  target  courses.  Competency  level  information  was 
obtained  from  the  lesson  objectives  of  each  target  course.  If  the  competency  information 
provided  by  the  lesson  objectives  were  unclear,  course  administrators  were  contacted  for 
clarification. 

Readability  is  based  on  the  Flesch  Reading  Ease  Score,  which  is  determined  by 
the  structure  of  words  and  sentences  (Flesch,  1991).  Its  scale  ranges  from  0  to  100.  The 
higher  the  score,  the  easier  it  is  to  read.  To  collect  this  data,  three  modules  were 
randomly  chosen  from  each  target  course.  Within  each  module,  groups  of  approximately 
200  words  were  copied  into  Microsoft  Word.  Then,  the  Microsoft  Word  readability 
statistics  tool  was  turned  on  and  run  on  the  group  of  words.  The  average  score  of  the 
three  modules  was  then  recorded  as  the  course’s  overall  readability  score. 

Some  e-learning  courses  require  100-percent  mastery,  which  means  that  a  student 
must  correctly  answer  all  of  the  exercise  and  test  questio  ns  presented  within  the  course 
before  they  are  allowed  to  proceed.  For  instance,  DAU  students  get  three  attempts  to 
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obtain  100-percent  mastery  of  their  end-of-module  tests.  If  the  student  fails  on  the  third 
try  of  any  test,  they  are  locked-out  and  must  request  to  be  re-admitted  into  the  course.  If 
re-admittance  is  denied,  then  the  student  must  re-take  the  course,  starting  from  the  very 
first  module.  Most  e- learning  courses  that  do  not  require  100-percent  mastery  eventually 
provide  students  with  the  correct  answers  after  the  student  has  made  several  failed 
attempts.  For  purposes  of  this  study,  “1”  identifies  courses  that  require  100-percent 
mastery,  while  “0”  identifies  courses  that  do  not  require  100-percent  mastery.  Table  3 
reveals  the  target  course  difficulty  characteristics. 


Table  3:  Distinguishing  Characteristics  of  Target  Course  Difficulty 


Course  Name 

Mastery 

Competence 

Level 

Readability 

Current  Topics  in  Financial  Management 

0 

1 

29.8 

Weapons  Systems  Pollution  Prevention 

0 

2 

19.9 

Advanced  Concept  Technology  Demonstrations 

0 

2 

36.7 

Integrated  Product  Support 

0 

1 

35.6 

Modification  Management 

0 

2 

22.6 

Acquisition  Logistics  Fundamentals 

1 

1 

** 

Fundamentals  of  Systems  Acquisition  Management 

1 

1 

** 

Reliability  and  Maintainability 

1 

2 

41.8 

Note.  **  unable  to  obtain.  For  Mastery;  “0”  =  100%  mastery  not  required  and  “1”  =  100%  mastery 
required.  For  Competence  Level:  “1”  =  knowledge  and  “2”  =  comprehension 
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Course  Feedback 


Feedback  was  measured  in  terms  of  interactivity  level  between  student  and 
course,  and  the  amount  of  “course  progression”  information  the  student  received  from 
both  the  course  material  and  instructors.  The  interactivity  of  lessons  and  exercises  is 
rated  as  “5”  (high),  “3”  (medium),  or  “1”  (low)  depending  on  the  researchers  subjective 
observation  of  five  (5)  randomly  selected  modules  from  each  target  course.  For  example, 
a  course  that  has  many  interactive  lessons  will  be  rated  as  “5”,  while  a  course  that  is 
wordy  and  static  will  be  rated  as  “1 .” 

Progression  assessment  refers  to  the  amount  of  feedback  received  by  the  student 
from  course  exams,  tests,  exercises,  lessons,  and  instructors  that  provided  the  student 
information  on  their  performance  and  whether  or  not  they  were  on  track  to  complete  the 
course  in  the  time  given.  Progression  assessment  is  rated  similar  to  interactivity  level. 
Table  4  reveals  the  feedback  characteristics. 


Table  4:  Distinguishing  Characteristics  of  Target  Course  Feedback 


Course  Name 

Interaction 

Level 

Progress 

Assessment 

Current  Topics  in  Financial  Management 

1 

1 

Weapons  Systems  Pollution  Prevention 

3 

3 

Advanced  Concept  Technology  Demonstrations 

3 

3 

Integrated  Product  Support 

1 

3 

Modification  Management 

5 

5 

Acquisition  Logistics  Fundamentals 

3 

5 

Fundamentals  of  Systems  Acquisition  Management 

3 

3 

Reliability  and  Maintainability 

5 

5 

Note.  “1”  =  low,  “3”  =  medium,  and  “5”  =  high 
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The  mean  completion  rate  for  the  target  courses  was  78.1  percent.  The  Reliability 
and  Maintainability  and  Fundamentals  of  Systems  Acquisition  Management  courses  had 
the  highest  completion  rates  with  89.6  and  89. 1  percent,  respectively.  The  Integrated 
Product  Support  and  Advanced  Topics  in  Technology  Demonstrations  course  have  what 
this  study  considers  average  completion  rates  with  73.1  and  71.9  percent,  respectively. 
While  the  Weapons  Systems  Pollution  Prevention  and  Current  Topics  in  Financial 
Management  courses  had  the  lowest  completion  rates  with  66.6  and  64.7  percent, 
respectively.  The  completion  rates  of  these  latter,  two  courses  are  comparable  to  Jewett’s 
(1997)  findings  that  distance  learners  complete  on  average  only  66  percent  of  their 
courses. 

Instrument  Development 

The  first  phase  of  developing  the  ECQ  was  to  generate  a  pool  of  items 
capitalizing  on:  a)  conceptualizations,  statements,  and  suggestions  from  previous  attrition 
research  on  various  external  factors  likely  to  operate  in  e-  learning  environments 
(Catalano,  1985;  Miller,  1967;  Spanard,  1990);  b)  empirical  research  on  motivational 
theory  as  it  relates  to  activation,  direction,  intensity,  persistence,  and  termination  of  goal- 
directed  behavior  (Campbell  &  Pritchard,  1976;  Kanfer,  1990;  Lawler,  1994;  Steers  & 
Porter,  1991);  c)  notes  generated  from  interviews  with  e- learning  subject  matter  experts; 
d)  observations  emanating  from  unobtrusive  participation  in  an  e-  learning  course;  and  e) 
the  content  analysis  of  previous  e-  learning  course  critiques.  A  fundamental  assumption 
used  to  develop  the  instrument  is  that  all  people  who  sign  up  for  e-  learning  courses  are 
initially  motivated  to  complete  the  course.  Somewhere  along  their  e- learning  journey. 
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students  are  either  motivated  to  sustain  behavior  and  complete  the  course,  or  de¬ 
motivated  to  a  point  that  causes  them  to  terminate  behavior  and  therefore  drop  out  along 
the  way. 

The  items  include  rationale  for  engaging  with  e- learning,  intent  to  finish  the 
course,  and  motivational  traits.  The  ECQ  consisted  of  an  introduction  page  with 
instructions,  demographics  section,  and  23  items  used  to  collect  quantitative  data.  The 
demographic  section  requests  information  on  the  respondent’s  rank  or  grade,  marital 
status,  whether  they  have  children  or  not.  Items  also  requested  the  name  of  the  e- learning 
course  completion  status,  whether  or  not  the  student  requested  an  extension,  whether  they 
had  to  retake  the  course,  and  the  number  of  e-  learning  courses  previously  taken.  Three 
types  of  items  were  used:  check  all  that  apply,  choose  the  best  answer,  and  a  5-point 
Likert  scale  ranging  from  1  (Strongly  Disagree)  to  5  (Strongly  Agree),  with  3  being 
“indifferent.”  Four  of  the  items  are  subsets  of  validated  motivational  scales  used  by  Ray 
(1979),  Favor  (1982),  and  Elliot  and  Church  (1997).  The  other  items  were  derived  from 
previous  research  in  the  field  of  education,  and  interview,  content  analysis,  and 
observation  transcript  notes. 

The  second  phase  of  instrument  development  involved  instrument  validation.  To 
do  this,  the  instrument  was  provided  to  subject  matter  experts  practicing  in  the  disciplines 
of  education  and  psychology.  They  analyzed  the  ECQ  for  validity  and  consistency  while 
looking  for  and  eliminating  any  contamination.  The  validation  phase  resulted  in  a 
validated  instrument  containing  23  items.  The  ECQ  was  then  transformed  into  a  web 
page  using  Cold  Fusion  programming  language  software.  It  was  then  hosted  on  the  Air 
Force  Institute  of  Technology  (AFIT)  School  of  Engineering  and  Management  Web 
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Server  and  given  an  address  of  http://en.afit.edu/env/elearning.  The  ECQ  was  now 
classified  as  an  on-line  (i.e.  web-based)  data  collection  instrument  (Dooley,  2001:177). 

Once  the  ECQ  was  developed,  the  third  phase  of  development  was  to  submit  the 
survey  and  justification  the  Air  Force  Survey  Branch  (AFPC/DPSAS)  for  approval.  The 
ECQ  was  approved  on  13  December  2001  with  a  Survey  Control  Number  ofSCN  01-120 
and  an  expiration  date  of  30  March  2002.  The  SCN  granted  authority  to  randomly  select 
and  administer  the  ECQ  to  Air  Force  personnel  based  on  the  prearranged  agreement. 

The  use  of  a  web-based  questionnaire  instead  of  a  paper-based  questionnaire 
appears  to  be  the  most  appropriate  way  to  quantitatively  collect  data  for  this  research 
effort.  Previous  research  tends  to  agree.  According  a  recent  web-based  versus  paper- 
based  survey  study,  web-based  surveys  that  have  non-sensitive  content  are  just  as  valid 
and  reliable  as  paper-based  questionnaires  (Franke,  2001 :53).  The  ECQ  contains  no 
sensitive  content.  Therefore,  it  appears  to  be  an  appropriate  technique  to  collect  data. 

The  ECQ  is  presented  in  its  entirety  in  Appendix  C. 

Subject  Pool 

The  subjects  used  in  this  experiment  were  active  duty  Air  Force  members,  civil 
service  employees,  or  contractors  working  for  the  Air  Force  at  U.S.  Air  Force 
installations  around  the  world.  The  subject  pool  was  heterogeneous  consisting  of  both 
male  and  female,  with  ranks  ranging  from  Lieutenant  Colonel  to  Airman  Basic  for 
military  members,  and  pay  grades  ranging  from  GS-14  to  GS-4  for  civilians.  To  qualify 
as  a  subject  for  this  experiment,  each  subject  had  to  meet  all  of  the  following 
prerequisites: 
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1.  An  Air  Force  member  or  working  for  the  Air  Force  in  that  status, 

2.  Previously  enrolled  in  one  of  the  target  courses  between  the  months  of  May 
thru  December  2001, 

3.  Had  registered  for  the  course  using  a  military  email  address  instead  of  a 
personal  email  address  (i.e.  name@basename. not 
name@hotmail.com),  and 

4.  Not  an  e-learning  designer,  instructor,  or  administrator  assigned  to  AFIT  or 
DAU. 

A  list  of  potential  subjects  from  each  target  course  was  extracted  from  both  the 
Virtual  Schoolhouse  database  and  the  DAU  Operational  Support  System  database.  The 
listings  included  name,  e-mail  address,  grade/rank,  unit,  and  course  completion  status 
(i.e.  completed  or  withdrew).  Subjects  were  then  randomly  selected  from  the  lists.  All 
duplicate  names  (i.e.  subjects  that  had  enrolled  in  more  than  one  of  the  target  courses 
during  the  experimental  period)  were  eliminated.  Eliminating  duplicate  names  ensured 
that  each  subject  received  the  ECQ  only  once. 

All  subjects  who  participated  in  this  survey  did  so  voluntarily  and  did  not  receive 
any  type  of  compensation  for  participation.  Subjects  were  informed  that  the  individual 
results  gleaned  from  their  participation  would  be  combined  with  the  responses  of  other 
subjects  who  have  taken  the  same  course,  as  well  as  compared  to  subject  responses  from 
different  e-learning  courses.  Subjects  were  also  informed  that  the  results  would  be 
provided  to  the  instructors,  administrators,  and  designers  of  the  e- learning  courses  used  in 
this  experiment. 
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Data  Collection  Procedures 


As  stated  earlier  in  this  chapter,  interviews,  content  analysis,  and  observations 
were  conducted  to  gather  data  for  this  research.  Handwritten  notes  were  taken  during 
each  interview  on  the  subject  matter  expert’s  responses  to  interview  questions,  as  well  as 
on  other  pertinent  and  unfo  reseen  discussion  that  occurred.  A  historical  transcript  was 
generated  during  the  OSS&E  e-learning  course  observation.  In  addition  to  taking  notes 
on  each  module  event,  the  transcript  included:  date,  time,  and  place  the  observation  was 
conducted;  problems  encountered;  level  of  interaction;  response  times;  motivational 
level;  and  number  of  times  interrupted  while  engaged  with  module.  Information  was 
collected  from  the  six  e-  learning  course  critiques  by  reviewing  each  one  then  highlighting 
similar  or  recurring  responses.  After  all  the  critiques  were  reviewed  and  highlighted,  the 
data  was  compiled  into  a  table  (Appendix  B)  that  summarized  the  content  analysis 
findings. 

To  gather  quantitative  data,  924  email  messages  (Appendix  C)  were  sent  out  to 
the  randomly  selected  subjects  asking  them  to  participate  in  the  study.  A  follow-up  email 
message  (Appendix  D),  asking  those  who  did  not  initially  participate  to  reconsider,  was 
sent  seven  days  later.  The  messages  explained  the  purpose  of  the  ECQ,  how  subject 
names  were  selected,  and  provided  a  link  to  the  website  where  the  ECQ  was  located.  All 
email  messages  were  batch  processed  by  target  course.  In  other  words,  all  of  the  subjects 
who  had  taken  the  Modification  Management  course  received  an  email  message 
identifying  only  that  course  as  the  one  in  question.  This  was  done  to  jog  the  subject’s 
memory  in  case  they  had  forgotten  which  course  they  had  previously  taken.  It  also 
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provided  subjects  with  guidance  to  select  that  course  from  the  drop-down  menu  on  the 
ECQ’s  demographics  information  page.  Selecting  the  proper  course  was  important 
because  it  allowed  statistical  information  to  be  differentiated  by  target  course,  instead  of 
simply  being  generalized  to  the  domain  of  e-  learning. 

Subjects  were  then  asked  to  click  on  the  link  and  take  the  questionnaire.  The  link 
opened  up  the  ECQ  introduction  page  that  provided  a  short  greeting,  questionnaire 
instructions,  and  a  “Start  Survey”  button.  User  responses  to  the  ECQ  consisted  of  “point- 
and- click”  and  typed  operations.  Though  the  subjects  were  asked  to  provide  some 
demographic  information,  a  couple  of  steps  were  taken  to  protect  their  anonymity.  First, 
they  were  not  asked  to  provide  their  name,  age,  race,  gender,  or  unit  at  any  time.  Second, 
once  the  respondent  completed  the  survey,  he  or  she  was  asked  to  click  the  “Finish” 
button.  When  the  Finish  button  was  pressed,  all  response  data  was  sent  and  saved 
directly  to  an  Access  2000  database  that  had  no  way  of  determining  from  whom  the 
information  was  being  sent.  Information  as  to  subject  anonymity,  and  steps  taken  to 
protect  it,  was  provided  on  the  introduction  page. 

The  final  page  of  the  ECQ  was  a  screen  thanking  subjects  for  their  participation. 
The  screen  also  provided  an  email  link  in  case  the  subject  wanted  to  contact  the 
researcher.  To  protect  against  receiving  blank  responses,  the  ECQ  programming  code 
performed  error- checking  so  blank  items  were  not  allowed.  In  addition,  the  code  saved 
the  date,  time,  and  Internet  Protocol  address  to  the  corresponding  Access  record.  This 
information  was  used  to  identify  and  eliminate  multiple  responses  from  the  same  person, 
or  responses  that  were  perceived  as  bogus. 
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The  questionnaire  was  kept  on-line  for  12  days  after  the  follovv^up  email  message 
was  sent  out.  In  the  end,  497  usable  responses  were  received,  for  an  overall  response  rate 
of  58.9%.  Four  hundred  and  sixty  four  (93.4%)  of  the  respondents  stated  they  completed 
their  course,  while  33  (6.6%)  stated  they  had  to  withdraw.  Therefore,  non-response  bias 
could  have  been  a  factor  because  most  of  the  responses  received  were  from  subjects  that 
had  actually  completed  their  e-  learning  course. 

Statistical  Analysis 

Three  statistical  techniques  were  employed  in  this  study.  Under  the  assumption 
of  normality,  a  statistical  analysis  technique  called  the  Independent  Samples  t-Test  was 
chosen  for  analyzing  factors  of  motivation  in  all  the  Likert-type  ECQ  items.  The  t-Test 
compares  the  mean  scores  of  two  groups  on  a  given  variable.  The  two  groups  act  as  the 
independent  variable  (factor),  while  the  given  variable  acts  as  the  dependent  response.  In 
this  case,  the  two  groups  are:  Completed  and  Dropped  students.  Levene’s  Test  for 
Equality  of  Variances  is  used  in  conjunction  with  the  t-Test  to  analyze  whether  the  two 
groups  have  approximately  equal  variance  on  the  dependent  variable  (Levene,  1960). 
Failing  to  reject  the  null  hypothesis  implies  that  there  is  no  statistical  difference  in 
completion  status  classifications  for  the  given  factor  of  motivation 

All  of  the  ECQ  items  that  collected  frequency  responses  (i.e.  Choose  best  answer 
and  Check  all  that  apply)  required  a  Chi-square  (x^)  test  for  independence.  The  test 
for  independence  was  used  to  determine  if  each  factor  of  motivation  and  if  the  course 
characteristics  were  independent  of  completion  status  (Completed  versus  Dropped). 
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Using  the  probability  that  a  motivational  factor  is  selected  from  the  ECQ  as  an  example, 
the  null  hypothesis,  that  the  classifications  are  independent,  is  represented  by: 

Ho:  P(S)  =  P(S|C)  =  P(S|D)  (2) 

P(S)  is  probability  of  selecting  the  item.  P(S|C)  and  P(S|D)  are  conditional 
probabilities  of  selecting  the  item  given  that  the  respondent  completed  or  dropped  out  of 
the  course.  Failing  to  reject  the  null  hypothesis  implies  there  is  no  statistical  difference 
between  completion  status  classifications  for  the  given  factor  of  motivation.  The 
alternative  hypothesis,  then,  is  that  relative  completion  of  the  e-  learning  course  does 
matter,  and  at  least  one  of  the  conditional  probabilities  is  different  from  the  others.  Each 
factor  has  a  similar  null  hypothesis,  and  the  test  for  independence  was  employed  to 
determine  whether  the  pattern  of  conditional  probabilities  in  the  data  are  unlikely,  given 
the  null  hypotheses  are  true. 

Hypothesis  5  required  an  interaction  effect  of  success  orientation  and  completion 
status  on  the  course  characteristics  of  average  time  per  module  and  time  given  to 
complete  the  entire  course.  The  interaction  effect  used  is  a  two-way  analysis  of  variance 
(ANOVA)  to  determine  if  there  is  statistical  significance  between  success  orientation  and 
completion  rate  (first  factor)  on  the  course  length  characteristics  (second  factor). 
Therefore,  there  are  three  null  hypotheses  associated  with  the  two-way  ANOVA:  the 
means  of  the  first  factor  are  equal,  the  means  of  the  second  factor  are  equal,  and  there  is 
no  interaction  between  the  factors.  An  U-test  is  used  to  determine  significance  between 
the  factors. 
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All  statistical  analyses  are  reported  in  Chapter  IV  with  tables  and  graphs.  The 
goal  was  to  find  statistically  significant  similarities  in  responses  across  the  completed  and 
dropped  groups,  as  well  as  assess  the  hypotheses  presented  in  Chapter  11. 

Summary 

This  ehapter  explained  the  research  approach  and  methodology  used  to  eompare 
and  eontrast  expbratory  findings  with  questionnaire  (ECQ)  responses  from  497  randomly 
selected  Air  Force  personnel.  The  research  goal  was  to  assess  those  motivational  factors 
that  influence  e- learning  course  completion  rates  and  either  support  or  refute  emerging 
theory.  Completion  rate  (Ecr)  data  was  used  in  conjunction  with  target  eourse 
characteristics  and  ECQ  responses  to  determine  if  any  differences  were  more  or  less 
likely  for  either  complete  or  drop  out  groups  based  on  motivational  eonstructs.  The 
results  of  all  the  analysis  and  assessments  will  then  be  used  to  draw  eonclusions  about  the 
impact  that  motivation  has  on  e-  learning  course  completion  rates. 
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IV.  Results 


“Trust — but  verify.  ” 

Ronald  Reagan  (1989) 


Introduction 

The  intent  of  this  research  was  to  ident  ify  and  measure  those  motivational  factors 
that  influence  the  desire  to  complete  e- learning  courses.  This  chapter  presents  the 
analysis  and  findings  of  the  E-  learning  Course  Questionnaire  (ECQ)  response  data. 
Response  data  will  be  compared  and  analyzed  along  with  exploratory  data  to  support  or 
refute  the  six  hypotheses  presented  in  chapter  two.  First,  response  demographics  data  are 
summarized.  Next,  the  focus  shifts  to  answering  the  research  questions  and  associated 
hypotheses  statements  using  the  data  acquired  from  the  ECQ.  Finally,  the  chapter  will 
review  the  additional  comments  made  on  the  ECQ. 

Respondent  Demographics  Data 

The  purpose  of  the  first  section  of  the  questionnaire  (ECQ)  was  to  gather 
demographics  data  about  the  respondents.  The  demographic  data  collected  included 
military  rank,  civilian  pay  grade,  marital  status,  whether  they  had  children  or  not,  target 
course  enrolled  in,  completion  status,  number  of  prior  e- learning  courses  taken,  and 
whether  or  not  they  had  to  retake  the  course  or  extend  their  time  limit.  Each  of  these 
demographic  variables  could  directly  or  indirectly  influence  a  student’s  ability  or  desire 
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to  complete  an  e-  learning  course.  Therefore,  it  is  important  to  see  if  there  are  correlations 
between  the  demographic  variables  and  e-  learning  course  completion  rates. 

Completion  Status 

The  questionnaire  asked  respondents  to  state  whether  they  completed  or  did  not 
complete  the  course  in  which  they  were  enrolled.  Out  of  the  497  responses,  a  total  of  464 
(93.4%)  stated  they  successfully  completed  the  course,  while  33  (6.6%)  stated  they  did 
not  complete  the  course.  Those  that  did  not  complete  the  course  will  be  identified  as 
“dropped”  throughout  the  remainder  of  this  chapter.  Though  261  (52.5%)  of  the 
respondents  were  civilians,  and  236  (47.5%)  were  military  members,  25  (75.8%)  of  the 
33  that  dropped  were  civilians.  Fifty-one  (11%)  of  the  respondents  that  completed  the 
course  had  to  either  retake  the  course  or  request  an  extension.  On  the  other  hand,  29 
(87.9%)  of  the  33  that  dropped  had  at  one  time  or  another  retook  the  course  or  requested 
an  extension.  The  information  received  on  those  that  dropped,  however,  does  not  clarify 
whether  they  dropped  out  while  retaking  the  course,  or  during  the  extension.  Information 
on  completion  status  is  illustrated  in  Table  5. 


Table  5:  Completion  Status  Distribution 


Total 

Percent 

Completed 

Percent 

Dropped 

Overall 

497 

93.4% 

6.6% 

Military 

236 

96.6% 

3.4% 

Civilians 

261 

90.4% 

9.6% 

Had  to  Retake  Course 

22 

40.9% 

59.1% 

Requested  an  Extension 

58 

72.4% 

27.6% 

62 


Rank  and  Grades 


The  majority  of  the  civilians  (162)  were  between  the  pay  grades  of  GS-11  and 
GS-14.  This  accounted  for  36.2%  of  the  total  responses  received.  For  pay  grades  of  GS- 
1  through  GS-10,  85  responses  were  received,  which  is  17.1%  of  the  total  responses 
received.  Contractors  (14)  accounted  for  the  remaining  2.8%  of  the  civilian  respondents. 
The  pay  grade  of  GS-14  was  the  highest  civilian  response  received.  Of  the  military 
members,  officers  accounted  for  172  (34.6%)  of  the  responses,  followed  by  64  (12.9%) 
enlisted  members.  The  highest  military  rank  that  responded  to  the  ECQ  was  Colonel.  A 
breakdown  of  the  respondents  by  rank  and  grade  are  illustrated  in  Figures  8  thru  10. 

LtCol(2%)  Col  (1.2%) 


Capt(9.9%)  lLt(4.4%) 

Figure  8:  Officer  Rank  Distribution  (percent  of  total) 


CMSgt(1.4%)  SSgt(0.6%) 


MSgt  (5.6%) 

Figure  9:  Enlisted  Rank  Distribution  (percent  of  total) 
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Contractor  (2.8%)  GS  1-10 


GS  11-14  (32.6%) 

Figure  10:  Civilian  Grade  Distribution  (percent  of  total) 


Marital  and  Children  Status 

The  information  gathered  on  marital  status,  and  whether  or  not  the  student  has 
children,  is  important  because  it  could  possibly  identify  those  students  having  more 
personal  demands  (i.e.  off-task  demands)  than  others.  Spanard  (1990)  found  that 
persisting  students  had  less  familial  pressure  than  non-persisting  students.  Therefore,  a 
student’s  family  situation  could  prove  instrumental  in  determining  whether  they  persist  or 
drop  out  of  an  e- learning  course. 

Over  half  of  the  respondents  (290)  were  married- vwY/i- children.  They  accounted 
for  58.4%  of  the  total  responses  received.  Those  single-wit/iowt-children  were  the  second 
largest  group  (94),  comprising  18.9%  of  the  total  responses  received.  Those  married- 
children  (72)  accounted  for  14.5%,  while  the  single- children  respondents 
(41)  accounted  for  the  remaining  8.2%.  ECQ  response  data  revealed  that  those  students 
single-w(Y/(OMt-children  had  the  lowest  percentage  of  dropouts  (3.2%).  The  single-w/tA- 
children  group,  however,  had  the  highest  percentage  of  dropouts  (17.1%).  The  married- 
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w/rA- children  and  married- wzYAow^-children  groups  had  drop  out  rates  of  6.9%  and  4.2%, 
respectively.  Findings  are  illustrated  in  Table  6. 


Table  6:  Marital  and  Children  Status  Distribution 


Marital  Status 

Total 

Percent 

Completed 

Percent 

Dropped 

With  Children 

290 

93.1% 

6.9% 

s 

Without  Children 

72 

95.8% 

4.2% 

With  Children 

41 

82.9% 

17.1% 

To 

m 

Without  Children 

94 

96.8% 

3.2% 

These  findings  suggest  more  single- children  respondents  dropped  out  than 
any  other  group  within  the  marital  status  and  children  category.  The  single- wit/?- children 
group  seem  to  be  different  to  the  other  three.  There  does  not  appear  to  be  a  difference 
between  married- w/t/i- children,  married- vr/t/ioMt-children,  and  single-iv/t/iowt-children 
groups.  Therefore,  these  three  groups  were  combined  and  compared  to  the  single- wiY/i- 
children  group.  A  statistically  reliable  influence  of  marital  status  on  e-learning  course 
completion  rates  (%  =  7.85,/?  <  .01)  was  discovered.  The  conditional  probability  of 
completing  the  course  and  being  single- w/Y/i- children  was  .85  while  the  probability  of 
completing  the  course  and  not  being  single- w/Y/i- children  was  .94. 

E-learning  Experience 

Approximately  half  of  the  respondents  for  this  research  had  no  prior  e- learning 
experience  (48.1%).  There  were  110  (22.1%)  respondents  that  had  taken  only  one  prior 
e-learning  course.  This  proved  to  be  the  second  largest  experience  category,  followed  by 
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46  (9.9%)  respondents  who  had  taken  two  prior  e- learning  courses.  The  “other”  category 
varied  from  three  to  35  prior  courses,  with  anywhere  from  37  (7.4%)  respondents  to  one 
(0.2%)  respondent  claiming  to  had  taken  that  particular  number  of  prior  courses.  The 
distribution  of  respondents  for  the  six  highest  levels  (i.e.  none  thru  five)  of  prior  e- 
learning  experience  is  illustrated  in  Figure  1 1. 


Figure  11:  Prior  E-Learning  Experience  (by  number  of  courses) 

Hypothesis  Testing 

The  second  part  of  the  questionnaire  asked  the  respondents  to  select  items  within 
each  statement  based  on  their  e-learning  experiences  with  the  target  course  in  question. 
As  mentioned  earlier  in  this  chapter,  ECQ  statements  were  presented  in  one  of  three 
forms:  5-point  Likert-scale,  Choose  best  answer,  or  Check  all  that  apply.  The  following 
sections  discuss  the  response  and  course  characteristics  data  used  to  test  the  hypotheses. 
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The  Influence  of  External  Factors 

During  the  period  between  May  and  December  2001,  the  majority  of  the 
respondents  (75%)  engaged  with  e- learning  at  woric  during  regular  business  hours.  More 
than  half  of  these  individuals  stated  they  encountered  technical  problems  while  taking  the 
course.  Table  7  shows  the  probability  that  technical  problems  influenced  their  decision  to 
complete  or  drop  out  of  the  course. 

Table  7:  Comparison  of  Technical  Problems  Encountered  between  Completed  and  Dropped 

Groups 


External  Factor:  Technical  Problems 

P(S) 

P(S|C) 

P(S|D) 

Slow  or  Choppy  System  Response 

.22 

.20 

.39 

6.48** 

Network  Outages 

.20 

.19 

.33 

3.70* 

Hardware/Software  Problems 

.16 

.15 

.27 

3.27 

Note.  A^=497.  *  /?<.05,  **  p<.01.  P(S)  probability  of  selecting  item.  P(S|C) 
and  P(S|D)  probability  of  selecting  item  given  completed  or  dropped  course. 


The  conditional  probability  of  encountering  the  technical  problems  of  slow  and 
choppy  system  response  times,  and  network  outages,  given  that  the  respondents 
completed  or  dropped  out  of  the  course  supports  ///„.  The  influence  of  hardware  and 
software  problems  on  completion  status  was  not  statistically  reliable. 

Other  distractions  beyond  technical  problems  are  off- tack  demands.  Table  8 
indicates  the  consequences  of  off-task  demands  on  completion  rates.  Noise  was  the  only 
off-task  demand  whose  influence  on  completion  rates  was  statistically  reliable,  but  not  in 
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the  direction  hypothesized.  Job  demands  was  the  most  commonly  encountered  off- task 
demand  while  engaged  with  e-  learning,  but  its  influence  on 


Table  8:  Comparison  of  Off-Task  Demands  Encountered  between  Completed  and  Dropped 

Groups 


External  Factor:  Off- Task  Demands 

P(S) 

P(S|C) 

P(S|D) 

Job  Demands 

.62 

.61 

.67 

0.36 

Noise 

.47 

.48 

.30 

4.09* 

Personal  Demands 

.10 

.09 

.18 

2.94 

Note.  N=A91.  *  p<.05,  **  p<.01.  P(S)  probability  of  selecting  item.  P(S|C)  and 
P(S|D)  probability  of  selecting  item  given  completed  or  dropped  course. 


completion  rates  was  not  statistically  reliable.  When  the  respondents  were  asked,  “What 
caused  or  triggered  you  to  stop  working  on  the  course?” — three  out  of  three  that  chose 
“had  to  go  TDY”  dropped  out,  and  eleven  out  of  fourteen  that  chose  “other  demands 
became  more  important”  also  dropped  out.  The  differences  between  the  conditional 
probabilities  could  not  be  tested  using  a  contingency  table  because  of  low  response 
frequencies  in  the  “complete”  category  (0  for  “TDY”  and  3  for  “other  demands...”). 
Brightman  (1999)  states  that  contingency  table  analysis  should  not  be  conducted  when 
any  category  has  a  frequency  below  five  (5).  Even  though  no  statistical  test  results  could 
be  shown  for  all  the  data,  Hjh  seems  to  be  partially  supported.  The  problem  could  rest 
with  the  items  chosen  to  test  Hjt,.  They  could  have  been  too  general  in  nature,  and 
therefore  poorly  represented  how  off- task  demands  actually  affect  completion  rates. 
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An  independent  groups  t-test  was  performed  comparing  the  mean  likeability 
rating  for  the  completed  group  {m  =  3.80,  sd  =  0.97)  with  that  for  the  dropped  group  (m  = 
2.90,  sd  =  1.31)  regarding  environmental  support.  The  alpha  level  was  .05.  This  test  was 
found  to  be  statistically  significant  {t  =  -4.97,/?  <  .0001),  indicating  those  respondents 
that  received  more  environmental  support  were  less  likely  to  drop  out  of  the  course  than 
those  respondents  that  received  little  or  no  environmental  support.  In  addition,  a 
comparison  of  actual  versus  expected  frequencies  according  to  whether  or  not  the  course 
was  “required”  shows  a  statistically  reliable  influence  of  completion  group  (x^  =  92.56,/? 
<  .0001).  The  conditional  probability  of  completing  a  required  course  was  .89  while  the 
probability  of  completing  a  non-required  course  was  .73.  Therefore,  there  is  strong 
support  for  Hu- 

Assessing  Intrinsic  and  Extrinsic  Motivation 

Results  show  that  the  majority  of  the  respondents  (81%)  enrolled  in  e- learning  to 
gain  knowledge.  Even  more  than  that  (85%)  found  the  convenience  of  “any  time” 
learning  the  most  appealing  factor  about  e-  learning.  The  data  indicate  minor  differences 
on  intrinsic  motives  between  respondents  who  completed  or  dropped  out  of  the  e- learning 
course.  No  differences  were  found  for  the  three  need  for  competence  items  (Table  9). 

The  conditional  probability  of  the  respondents  having  a  need  for  competence  given  that 
they  completed  or  dropped  out  of  the  course  does  not  supports  Hza- 

There  were  differences,  however,  for  two  of  the  five  need  for  self-determination 
items.  Table  10  shows  the  probability  that  the  intrinsic  component  of  self-determination 
influenced  the  respondent’s  decision  to  complete  or  drop  out  of  the  course.  One 
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interesting  finding  was  the  respondents  that  selected  two  or  less  of  the  need  for  self- 
determination  items  have  higher  drop  out  rates,  while  those  that  selected  three  or  more 
appeared  to  have  relatively  high  completion  rates. 

Table  9:  Comparison  of  Need  for  Competence  between  Completed  and  Dropped  Groups 


Intrinsic  Factor:  Need  for  Competence 

P(S) 

P(S|C) 

P(S|D) 

To  Gain  Knowledge 

.81 

.81 

.85 

0.36 

Improve  Job  Performance 

.62 

.61 

.76 

2.77 

Get  Some  Specific  Information 

.13 

.13 

.21 

2.06 

Note.  N=A91.  *  /?<.05,  **  p<.01.  P(S)  probability  of  selecting  item.  P(S  C)  and 

P(Sp)  probability  of  selecting  item  given  completed  or  dropped  course. 

Table  10:  Comparison  of  Self-Determination  between  Completed  and  Dropped  Groups 

Intrinsic  Factor:  Self-Determination 

P(S) 

P(S|C) 

P(S|D) 

X^ 

Convenience  of  “Any  Time”  Learning 

.85 

.86 

.70 

6.63** 

Ability  To  Fit  Into  Schedule 

.77 

.79 

.48 

15.68** 

Could  Work/Leam  At  Own  Pace 

.77 

.78 

.70 

1.22 

Could  Work/Leam  Independently 

.66 

.67 

.55 

2.07 

Convenience  of  “Anywhere”  Learning 

.52 

.53 

.48 

0.21 

Note.  N=A91.  *  /7<.05,  **  p<.01.  P(S)  probability  of  selecting  item.  P(S|C)  and 
P(S|D)  probability  of  selecting  item  given  completed  or  dropped  course. 
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The  conditional  probability  of  the  respondents  having  self-determination  needs 
given  that  they  completed  or  dropped  out  of  the  course  somewhat  supports  7/26  •  The 
convenience  of  “any  time”  learning  and  the  ability  to  fit  e-  learning  into  their  schedule 
strongly  supports  the  hypothesis,  while  none  of  the  other  self-determination  items  were 
statistically  reliable. 

Table  1 1  indicates  the  consequences  of  extrinsic  factors  on  completion  rates.  The 
conditional  probability  of  the  respondents  having  extrinsic  motives  given  that  they 
completed  or  dropped  out  of  the  course  somewhat  supports  //2c-  Accumulating  degree  or 
certificate  credits  supports  the  hypothesis,  while  seeking  a  promotion  was  not  statistically 
reliable.  These  findings  suggest  that  the  desire  to  complete  or  drop  out  of  an  e-  learning 
course  depends  on  the  extrinsic  reward. 


Table  11:  Comparison  of  Extrinsic  Factors  between  Completed  and  Dropped  Groups 


Extrinsic  Factor 

P(S) 

P(S|C) 

P(S|D) 

Accumulate  Degree/Certificate  Credits 

.37 

.39 

.21 

3.97* 

Promotion  Opportunity 

.19 

.19 

.15 

0.26 

Note.  A=497.  *  p<.Q5,  **  p<.01.  P(S)  probability  of  selecting  item.  P(S|C)  and 
P(S|D)  probability  of  selecting  item  given  completed  or  dropped  course. 


Achievement  Motives  and  Completion  Rates 

The  majority  of  the  respondents  viewed  themselves  as  being  highly  motivated 
(82%)  and  having  stronger  motives  to  achieve  success  (88%)  versus  avoid  failure.  Table 
12  shows  that  success  or  failure  orientation  had  no  influence  on  completion  rates. 
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Table  12:  Comparison  of  Success  Orientation  between  Completed  and  Dropped  Groups 


Success  Orientation 

P(S) 

P(S|C) 

P(S|D) 

Success-Oriented 

.86 

.86 

.88 

0.11 

Failure- Oriented 

.14 

.14 

.12 

0.11 

Note.  N=A91.  *  p<.Q5,  **  p<.01.  P(S)  probability  of  selecting  item.  P(S|C)  and 
P(S|D)  probability  of  selecting  item  given  completed  or  dropped  course. 


The  conditional  probability  of  success  orientation  given  that  the  respondent 
completed  or  dropped  out  of  the  course  does  not  support  H3.  In  addition,  results  from  an 
independent  groups  t-test  was  performed  comparing  the  mean  responses  of  the  completed 
group  (m  =  4.41,  sd  =  0.86)  with  that  for  the  dropped  group  {m  =  4.33,  sd  =  0.69)  as  to 
whether  they  planned  to  give  the  course  their  best  possible  effort  (Favor,  1982).  This  test 
was  not  statistically  significant  {t  =  -0.469,  p  =  .639),  indicating  that  there  is  no 
relationship  between  plans  to  give  the  course  their  best  possible  effort  and  whether  the 
course  gets  completed  or  not.  It  must  be  noted  that  self-serving  bias  could  have 
contaminated  these  results.  People  tend  to  perceive  themselves  favorably  when  asked 
self-assessment  questions  (Myers,  1996). 

Importance  of  Goal  Intensity 

The  data  indicates  that  those  respondents  who  enrolled  in  e-  learning  because  it 
was  required  for  their  job,  or  because  their  supervisor  recommended  it,  had  a  desire  to 
persist  longer  at  e-  learning  and  therefore  had  relatively  low  drop  out  rates.  A  comparison 
of  actual  and  expected  frequencies  shows  a  statistically  reliable  influence  of  the  goal 
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intensity  item  of  “job  requiremenf’  on  completion  rates  =  10.57,/?  <  .01).  The 
conditional  probability  of  completing  the  course  given  it  is  a  job  requirement  was  .96 
while  the  conditional  probability  of  completing  the  course  if  it  is  not  a  job  requirement  is 
.89.  In  addition,  111  out  of  1 15  respondents  that  enrolled  because  of  “supervisor 
recommendation”  completed  the  course.  The  differences  between  the  conditional 
probabilities  could  not  be  tested  using  a  contingency  table  because  of  low  response 
frequencies  in  the  “dropped”  category  (Brightman,  1999).  Even  though  no  statistical  test 
results  could  be  shown,  the  data  strongly  supports  H4. 

Interaction  Effect  of  Success  Orientation  on  Course  Length 

The  average  time  calculated  to  complete  a  target  course  module  ranged  from  .75 
hours  to  2.86  hours.  The  time  given  to  complete  the  target  courses  ranged  from  28  to  60 
days.  An  independent  t-test  was  performed  comparing  the  mean  likeability  rating  for  the 
completed  group  (m  =  1.28,  sd=QA2>)  with  that  for  the  dropped  group  {m  =  1.50,  sd  = 
0.50)  in  regards  to  average  time  per  module  was  found  to  be  statistically  significant  it  = 
2.16,  p  <  .01).  Time  given  to  complete  the  course  was  not  statistically  significance. 

Table  13  shows  the  interaction  effect  of  success  orientation  and  completion  status  on 
target  course  length.  Figures  12  and  13  show  the  same,  but  graphically. 

Course  length  characteristics  were  subjected  to  a  two-way  analysis  of  variance 
having  two  levels  of  orientation  (success  versus  failure)  and  two  levels  for  completion 
status  (completed  or  dropped).  For  an  alpha  of  .05,  the  interaction  effect  of  success 
orientation  and  completion  status  on  average  time  per  module  was  not  statistically 
reliable  (F(l)  =  .155, p  >  .05). 
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Table  13:  Interaction  Effect  of  Orientation  and  Completion  Status  on  Course  Length 


Success-  Oriented 

Failure- Oriented 

Average  Time  per  Module 

m  =  \  .29  sd  =  0.44 

m  =  1.24  5(7=  0.39 

(D 

n  =  398 

n  =  66 

o 

O 

Time  Given  to  Complete 

m  =  54.16v<i=  11 .25 

n  =  398 

m  =  54.06  sd=  11.90 

n  =  66 

Average  Time  per  Module 

m  =  53  sd=  0.51 

m  =  1.28  5(7=  0.32 

(U 

. 

n  =  29 

n  =  4 

& 

o 

Q 

Time  Given  to  Complete 

m  =  53.93  sd  =  9.54 

n  =  29 

m  =  55.5  5(7=  9.0 

n  =  4 

CD 


Orientation 
*  Success 
Failure 


Figure  12:  Interaction  Effect  of  Success  Orientation  and  Completion  Status  on  Average 

Time  per  Module 
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Orientation 


Success 

Failure 


Figure  13:  Interaction  Effect  of  Success  Orientation  and  Completion  Status  on  Time  Given 

to  Complete  Course 


Similarly,  the  interaction  effect  of  success  orientation  and  completion  status  on 
the  time  given  to  complete  the  course  was  not  statistically  reliable  (F(l)  =  .073,/?  >  .05) 
In  fact,  the  mean  for  the  failure- oriented  group  that  dropped  the  course  (m  =  55.5,sd  = 
9.0)  was  higher  than  the  mean  for  the  success- oriented  group  that  completed  the  course 
(m  =  55.5,  sd=  9.0). 

The  interactions  depicted  in  Figures  12  and  13  however  appear  consistent  with 
TTj.  Non-significance  may  be  due  to  relatively  small  sample  size.  The  denominator  of 
the  test  statistic  is  the  standard  error  (square  root  of  the  pooled  variance  divided  by 
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sample  size).  Increasing  sample  size  decreases  the  standard  error,  and  thus  increases  the 
magnitude  of  the  test  statistic.  The  graphic  shows  an  appearance  of  an  interaction  effect, 
but  given  the  size  of  the  sample,  we  cannot  be  sure  that  the  differences  are  statistically 
reliable.  Overall,  there  appears  to  be  something  going  on  here  that  warrants  further 
study. 


Self-Efficacy  Concepts  in  E-Learning  Environments 
Findings  from  the  ECQ  demographics  section  reveal  that  nearly  half  of  the 
respondents  (48%)  had  no  prior  e-  learning  experience.  Still,  there  was  a  strong  indication 
that,  as  the  respondents  started  the  course,  their  confidence  was  high  that  they  would 
complete  it  (response  m  =  4.58).  This  confidence  went  unchanged  for  80  percent  of  the 
respondents,  while  1 1  percent  stated  that  their  confidence  actually  increased.  A 
comparison  of  actual  and  expected  frequencies  shows  a  statistically  reliable  influence  of  a 
decrease  in  self-efficacy  on  completion  rates  =  45.30,/?  <  .01).  The  conditional 
probability  of  completing  the  course  given  a  decrease  in  self-efficacy  was  .06  while  the 
conditional  probability  of  dropping  out  of  the  course  given  a  decrease  in  self-efficacy  was 
.41.  These  findings  strongly  support  T/^.  However,  an  independent  t-test  was  performed 
comparing  the  mean  likeability  rating  for  the  completed  group  (m  =  4.60,  sd=  0.82)  with 
that  for  the  dropped  group  {m  =  4.18,  56?=  0.95)  in  regards  to  initial  confidence  that  the 
course  would  be  completed,  was  found  to  be  statistically  significant  {t  =  -2.824,/?  <  .01). 
The  conclusion  cannot  be  as  strong  as  initially  suggested  based  on  the  conditional 
probabilities.  The  respondents  who  did  not  complete  the  course  attributed  it  to  their  self- 
efficacy  getting  worse,  but  the  above  t-test  suggests  that  they  started  out  with  lower 
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efficacy.  In  short,  H(,  is  weakened  because  the  respondents  whose  confidence  decreased 
started  the  course  with  low  confidence  in  the  first  place. 

Exploring  the  Critical  Quality  of  Feedback 

The  respondents  indicated  that  the  majority  of  the  feedback  they  received  was  via 
electronic  messages  from  the  course  on  results  of  quizzes  and  exercises.  Table  14  shows 
the  probability  that  the  type  of  feedback  received  influenced  the  respondent’s  decision  to 
complete  or  drop  out  of  the  course. 


Table  14:  Comparison  of  Feedback  Type  between  Completed  and  Dropped  Groups 


Feedback  Type 

P(S) 

P(S|C) 

P(S|D) 

Electronic  Messages 

.92 

.94 

.67 

29.38** 

Instructor/Administrator  Messages 

.41 

.41 

.55 

2.50 

NO  Feedback  Received 

.20 

.20 

.18 

0.07 

Note.  N=A91.  *  /?<.05,  **  p<.01.  P(S)  probability  of  selecting  item.  P(S|C)  and 
P(S|D)  probability  of  selecting  item  given  completed  or  dropped  course. 


A  comparison  of  actual  and  expected  frequencies  shows  statistically  reliable 
influences  of  feedback  received  via  electronic  messages  on  the  complete  and  dropped 
groups.  Receiving  feedback  from  e-  learning  instructors  and  administrators,  or  receiving 
no  feedback  at  all,  did  not  have  a  statistically  reliable  influence  on  completion  rates.  An 
independent  samples  t-test  was  performed  comparing  the  mean  likeability  rating  for  the 
completed  group  (m  =  3.13,  sd=  1.00)  with  that  of  the  dropped  group  {m  =  2.70,  sd  = 
1.02)  with  regards  to  the  belief  that  there  was  high  levels  of  interactivity  with  the  e- 
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learning  course,  was  found  to  be  statistically  significant  {t  =  -0.43,/?  <  .05).  These 
findings  suggest  that  interactivity  level,  which  may  be  viewed  as  a  form  of  feedback, 
supports  Hja. 

In  addition,  an  independent  samples  t-test  was  performed  comparing  the  mean 
likeability  rating  for  the  completed  group  im  =  3.57,  sd  =  0.90)  with  that  for  the  dropped 
group  (m  =  3.03,  5(i=  1.10)  with  regards  to  the  belief  that  a  sufficient  amount  of  feedback 
was  received,  was  found  to  be  statistically  significant  {t  =  -3.269,/?  <  .01).  An 
independent  samples  t-test  was  performed  comparing  the  mean  likeability  rating  for  the 
completed  group  {m  =  3.72,  sd  =  0.84)  with  that  for  the  dropped  group  {m  =  3.27,  sd  = 
1.07)  in  regards  to  the  belief  that  any  feedback  received  was  timely,  was  also  found  to  be 
statistically  significant  (t  =  -2.914,/?  <  .01).  Therefore,  there  is  strong  support  for  Hja. 

There  was  no  significant  relationship  between  the  ability  to  assess  progress  while 
engaged  with  e-  learning  and  course  completion  rates  (/  =  - 1 .676,  p  >  .05).  Therefore,  Hjh 
was  not  supported. 

Attributing  Failure  to  Course  Difficulty 

Target  course  difficulty  characteristics  were  used  to  determine  whether  there  was 
a  significant  relationship  between  course  difficulty  and  completion  rates.  T-Test  results 
show  that  none  of  the  target  course  difficulty  characteristics — mastery,  competency  level, 
or  readability — had  a  statistically  reliable  relationship  with  completion  rates.  In  addition, 
no  respondent  selected  “The  course  was  too  difficult  for  me”  when  responding  to  the 
ECQ.  These  findings  suggest  that  course  difficulty  has  little  to  no  influence  on  a 
student’s  motivation  to  complete  or  drop  out  of  it.  Therefore,  Hg  was  not  supported. 
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Respondent  Comments 


The  last  section  of  the  ECQ  allowed  respondents  to  write  any  additional 
comments  about  their  e-  learning  experience.  Because  not  all  of  the  comments  can  be 
presented,  they  are  grouped  into  like  categories  and  summarized  below  in  Table  15. 


Table  15:  Summary  of  Additional  Comments 


Category 

Frequency 

of 

Comment* 

Proportion 

of 

Comments 

It  was  a  good  course 

40 

25.8% 

Hardware/Software  problems  hindered  e-  learning  efforts 

17 

11.0% 

Course  had  poor  or  static  content 

16 

10.3% 

Great  way  to  teach/learn 

15 

9.7% 

Job  demands/distractions  hindered  e-leaming  efforts 

11 

7.1% 

Network/Web  problems  or  restrictions  hindered  e- 

leaming  efforts 

9 

5.8% 

I  prefer  formal  teaching  over  e-  learning 

8 

5.2% 

Received  little  or  no  feedback  from  instructor 

7 

4.5% 

Note.  *  155  total  comments.  Some  respondents  made  more  than  one  (1)  comment. 


The  comments  seemed  to  differ  between  the  completed  and  dropped  group. 
Overall,  the  completed  group  appeared  to  be  satisfied  with  what  e- learning  had  to  offer. 
The  dropped  group  complained  mostly  about  the  hardware  or  software  problems  they 
encountered  while  trying  to  take  the  e-  learning  course.  Some  felt  as  though  the  technical 
problems  encountered  defeated  the  purpose  of  “any  time,  anywhere”  learning.  Other 

dissatisfied  respondents  commented  about  the  poor,  outdated,  or  boring  content  that  they 
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encountered.  As  put  by  one  respondent,  “While  it  (e-  learning)  is  convenient,  it  does  not 
motivate  me  to  learn  the  information.” 

Summary 

The  majority  of  the  respondents  successfully  completed  their  e- learning  course. 
Thus,  many  of  those  that  dropped  out  of  the  target  courses  between  May  and  December 
of  2001  chose  not  to  respond.  Maybe  the  same  factors  that  prevented  them  from 
completing  their  courses  in  the  first  place  are  still  present  and  prevented  them  from 
responding  to  the  questionnaire.  The  majority  of  those  respondents  that  did  choose  to 
participate  in  this  study  were  Air  Force  civilians,  in  the  grades  of  GS-11  thru  GS-14,  and 
married  with  children.  Table  16  summarizes  the  statistically  significant  factors  that 
positively  and  negatively  influence  e- learning  course  completion  rates. 

Table  16:  The  Positive  and  Negative  Influences  on  E-Learning  Course  Completion  Rates 


Positive  Influences 

Negative  Influences 

Environmental  Support 

Slow/Choppy  System  Response  Times 

Convenience  of  “Any  Time”  Learning 

Network  Outages 

Ability  to  Fit  Into  Schedule 

Lengthy  Modules 

Accumulate  Degree/Certificate  Credits 

Decrease  in  Self-efficacy 

Job  Requirement 

Initial  Confidence  (Self-efficacy) 

Electronic  Feedback  Messages 

High  Interactivity  with  Course 
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Table  17  contains  a  summary  of  the  hypotheses. 


Table  17:  Summary  of  Hypotheses 

Hypotheses 

Supported 

Hja'.  The  fewer  the  technological  problems,  the  more  likely  an  individual 

will  persist  at  e-  learning. 

Yes 

Hih'.  The  fewer  the  off- task  demands,  the  more  likely  an  individual  will 

persist  at  e- learning. 

Partial 

Hic'.  The  greater  the  environmental  support  the  more  likely  an  individual 

will  persist  at  e-  learning. 

Yes 

H2a'  The  greater  one’s  need  for  competence,  the  more  likely  they  will 

complete  the  e- learning  course. 

No 

H2h'  The  greater  one’s  need  for  self-determination,  the  more  likely  they 

will  complete  the  e-  learning  course. 

Partial 

H2c'  The  greater  one’s  extrinsic  motivation,  the  more  likely  they  will 

complete  the  e-  learning  course. 

Partial 

H3:  The  greater  an  individual’s  expectancy  for  success,  the  more  likely 

they  are  to  complete  an  e-  learning  course. 

No 

H4:  The  greater  the  goal  intensity  the  more  likely  the  e-  learner  will 

complete  the  e- learning  course. 

Yes 

Hs:  E- learning  course  length  in  terms  of  the  time  it  takes  to  complete  a 

module  and  the  total  time  given  to  complete  the  course,  will  have  different 

effects  on  completion  rates  for  success  and  failure  oriented  students. 

Partial 

H^:  The  greater  one’s  self-efficacy  in  e-learning  environments,  the  more 

likely  they  will  complete  the  e- learning  course. 

Yes 

Table  17  continued  on  next  page 
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Table  17  continued 


Hja'  E- learners  that  believed  they  received  sufficient  and  timely  feedback 

Yes 

information  are  more  likely  to  complete  e- learning  courses. 

Hjh'.  E- learning  courses  that  provide  feedback  information  that  shows 

progress  towards  proximal  and  distal  goals  will  ha\e  higher  completion  No 

rates  than  courses  that  do  not. 

Hs'.  The  greater  the  amount  of  external  difficulty  encountered  in  terms  of 

No 

course  design,  the  more  likely  the  e-  learner  will  drop  out  of  the  course. 


To  summarize,  five  of  the  thirteen  hypotheses  were  supported,  four  were  partially 
supported,  and  four  were  not  supported.  The  results  provide  overall  support  for 
application  of  the  integrated  motivational  model  in  an  e-learning  environment. 


82 


V.  Discussion 


“The  goal  of  shaping  information  technology  to  democratize  education  is  highly 
appealing,  but  there  are,  at  present,  no  strong  well-organized  forces  promoting  that 
end. " 


L.  Winner  (1998) 


Introduction 

Considering  the  enormous  potential  of  e-  learning,  and  the  huge  investment  the  Air 
Force  is  making  in  this  technology,  it  is  crucial  to  optimize  its  use.  Implementing  course 
designs  that  improve  e-  learning  course  completion  rates  is  a  step  in  the  right  direction. 

The  main  focus  of  this  study  was  to  identify  and  measure  external  and  motivational 
fectors  that  influence  a  person’s  desire  to  complete  or  drop  out  of  e- learning  courses.  By 
identifying  and  addressing  the  salient  objective  and  subjective  determinants  that  influence 
e- learner’s  desire  to  “go  the  distance,”  e-learning  course  designers  can  develop  new  on¬ 
line  courses  that  have  an  increased  probability  of  being  completed  (Rosenberg,  2001). 

Eight  (8)  e-  learning  courses  were  analyzed  along  with  497  questionnaire 
responses  to  answer  the  research  questions  below. 

Research  Question  1:  In  what  ways  do  technical  problems,  off-task  demands,  and 
environmental  support  (external  factors)  influence  motivation  to  complete  e- 
learning  courses? 

Research  Question  2:  How  does  e-learning  course  design  influence  the  effects  of 
external  factors  on  motivational  constructs? 

Research  Question  3:  What  motivational  factors  influence  e-learning  course 
completion  rates? 
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The  ensuing  sections  contain  discussions  of  the  findings,  to  include  a  review  of 
the  data  analysis.  These  discussions  are  followed  by  a  look  at  the  practical  and 
theoretical  implications  of  this  research.  The  next  two  sections  delve  into  some  noted 
research  limitations  and  recommendations  for  future  research.  The  final  section  of  this 
chapter  provides  an  overview  of  the  research  effort. 

Research  Question  1  Discussion 

Research  question  one  was  a  multipart  question  generated  from  information 
gathered  during  interviews,  observation,  and  critique  analysis.  Its  objective  was  to  reveal 
how  technical  problems,  off-task  demands,  and  environmental  support  (external  factors) 
influenced  motivation  to  complete  e-  learning  courses.  As  explained  in  the  literature 
review,  push-pull  theory,  presented  in  some  prior  retention  studies,  is  used  to  identify  the 
positive  and  negative  effects  that  external  factors  have  on  completion  rates.  In  short, 
push-pull  theory  argues  that  a  student’s  decision  to  stay  or  leave  the  learning  environment 
depends  on  the  combined  salience  of  all  forces  drawing  on  that  student’s  attention  and 
energy  along  with  the  costs  and  benefits  of  completing  the  course  (Catalano,  1985; 

Miller,  1967;  Spanard,  1990). 

Based  on  the  results  of  the  three  hypotheses  (///a,  Hu,,  and  Hic)  used  to  answer 
research  question  one,  it  was  determined  that  only  certain  technical  problems  and  off-task 
demands  (pull  factors)  decreased  e-  learning  course  completion  rates,  while  environmental 
support  (push  factor)  proved  to  be  a  highly  influential  factor  that  increased  e-  learning 
course  completion  rates.  Findings  revealed  that  slow  and  choppy  system  response,  and 
network  outages  negatively  influence  completion  rates,  even  though  respondents 
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indicated  these  technical  problems  occurred  less  than  20  percent  of  the  time.  More  than 
one-third  of  the  dropouts  identified  slow  and  choppy  system  response  and  network 
outages  as  the  most  unappealing  aspects  about  e-  learning.  These  findings  suggest  that  e- 
leaming  course  designers  must  design  e- learning  courses  to  meet  customer’s  needs, 
without  exceeding  the  customer’s  computer  network  limitations.  Realizing  that  any 
particular  e- learning  course  is  only  as  good  as  the  network  upon  which  it  rides,  e- learning 
designers  and  administrators  must  be  aware  of  such  limitations  and  either  reach  some  sort 
of  minimum  network  standard  agreement  with  customers,  or  be  very  sensitive  to  their 
future  needs  and  requests.  Still,  this  research  realizes  that  some  network  problems  are 
unavoidable  and  are  bound  to  impact  the  e-learning  experience.  Such  unexpected 
problems  can  and  should  be  planned  for  from  the  beginning. 

The  majority  of  the  respondents  indicated  their  displeasure  with  work-related 
demands  (meetings,  deadlines,  TDYs,  etc.)  encountered  while  they  were  engaged  with  e- 
leaming.  Noise,  however,  was  the  only  off-task  demand  that  significantly  influenced 
completion  rates,  but  surprisingly  in  a  positive  way.  The  probability  of  selecting  noise  as 
a  distraction  while  e-  learning  was  higher  for  those  that  completed  the  course  (.48)  than  it 
was  for  those  that  dropped  out  (.30).  Recall  that  people  are  limited  cognitively  (Kanfer  & 
Ackerman,  1989)  and  the  introduction  of  noise  that  constantly  causes  attention  and 
energy  to  be  focused  elsewhere  eventually  frustrates  the  student’s  motive  to  persistence. 
Researchers  argued  that  continued  fioistration  will  likely  lead  to  terminated  behavior 
(Spanard,  1990).  Respondents,  however,  indicated  that  noise  did  not  adversely  affect 
their  desire  to  persist  at  e-  learning.  As  interesting  as  they  are,  these  findings  suggest  that 
students  should  take  e- learning  away  from  work-related  distractions,  but  noise  positively 
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influences  their  desire  to  e- learn.  The  latter  contradicts  motivational  theory.  Nearly 
three-  fourths  of  the  respondents  engaged  with  e-  learning  at  work  during  regular  business 
hours.  If  this  is  indicative  of  most  e-  learners,  then  more  e-  learning  courses  should  be 
designed  to  allow  students  to  print  out  course  material.  The  student  could  then  print  out 
needed  material  and  re-locate  to  a  work- free  environment.  Re -location  may  help 
alleviate  work-related  distractions,  but  the  finding  on  noise  should  be  regarded  with 
caution. 

A  significant  difference  existed  between  respondents  that  received,  in  their  view, 
a  lot  of  environmental  support,  and  those  that  received  little  or  no  environmental  support. 
Course  completion  was  much  higher  for  those  respondents  that  felt  as  though  they 
received  a  lot  of  support.  In  contrast,  those  respondents  that  indicated  they  received  low 
or  inadequate  amounts  of  environmental  support,  dropped  out  significantly  more  often. 

As  expected,  those  respondents  required  to  take  e- learning  for  job  certification  purposes, 
received  more  environmental  support  than  those  respondents  who  took  “non- required” 
courses.  This  leads  to  the  belief  that  the  respondent’s  supervisor  and  peers,  realizing  the 
importance  of  job  certification,  provided  the  necessary  resources  (i.e.  time,  hardware, 
software,  etc.)  that  enabled  the  respondent  to  successfully  complete  the  course. 

Findings  also  suggest  that  the  respondent’s  familial  situation  impacted  completion 
rates.  Spanard  (1990)  found  that  persisting  students  have  greater  familial  support  and 
less  familial  pressure.  Findings  from  this  study  indicate  that  the  single-wiY/i-children 
respondents  dropped  out  three  times  more  than  the  other  three  marital  status  groups 
combined.  This  leads  to  the  belief  that  the  single-w/Y^-children  respondents  had  higher 
levels  of  familial  pressure  than  any  other  marital  status  group.  Reasons  could  abound. 
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but  single  parents  may  not  have  the  liberty  or  free  time  to  work  on  e-learning  during  non¬ 
work  hours  as  some  others  do. 

Even  though  there  is  no  proven  way  to  improve  the  relationship  between  a  student 
and  his  or  her  environment,  an  attempt  must  be  made  to  incorporate  environmental 
support  into  e-  learning  lesson  plans  and  course  designs.  These  findings  suggest  that  e- 
leaming  course  instructors  and  administrators  must  do  their  best  to  ensure  students 
receive  a  lot  of  environmental  support  while  taking  the  e- learning  course.  For  instance, 
e- learning  instructors  might  be  able  to  establish  some  sort  of  on-going  correspondence  or 
rapport  with  supervisors.  This  could  help  alleviate  some  misperceptions  about  e- learning 
and  provide  valuable  information  on  providing  good  support  for  successful  e- learning. 

Research  Question  2  Discussion 

Research  question  two’s  objective  was  to  determine  how  course  design  influenced 
the  effects  of  external  factors  on  motivational  constructs.  E-  learning  course  length, 
feedback,  and  difficulty  characteristics  were  used  to  identify  course  design 
characteristics.  Each  of  these  characteristics  was  tested  as  to  their  influential  effects  on 
motives  to  complete  e- learning  courses.  Based  on  the  results  of  the  four  hypotheses  {H5, 
Hja,  Hjh,  and  Hs)  used  to  test  this  question,  it  was  determined  that  average  time  per 
module,  course  interactivity  level,  and  receiving  feedback,  have  significant  effects  on  the 
desire  to  complete  e-  learning  courses.  The  course  difficulty  characteristics  of  mastery, 
competency  level,  and  readability  had  no  significant  effects  on  either  motivational 
constructs  or  completion  rates. 
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The  finding  that  average  time  per  module  had  a  significant  effect  on  e-  learning 
course  completion  rates  was  not  surprising.  It  was,  however,  surprising  that  the  length  of 
time  given  to  complete  the  course  was  not  significant.  This  may  have  been  because  the 
three  courses  with  the  longest  time  given  to  complete  them  were  all  “required”  courses. 
Statistical  analysis  reveals  that  respondents  tended  to  complete  the  required  courses  at 
much  higher  rates  than  the  non-required  ones.  Although  the  difference  between  time 
given  to  complete  e-  learning  courses  and  completion  status  did  not  prove  to  be 
statistically  significant,  the  thinking  here  is  that  longer  courses  leave  room  for  more 
technical  problems  and  off- tasks  demands.  Therefore,  it  may  promote  course  completion 
if  course  designers  consider  this  factor  when  designing  e-  learning  courses.  The  study 
also  revealed  that  the  interaction  effect  between  success  orientation  (success  or  failure) 
and  completion  status  on  course  length  was  not  statistically  significant.  Still,  the  graphics 
have  the  appearance  of  interaction.  Thus,  it  is  a  factor  that  may  warrant  further  attention. 

Respondents  indicated  that  high  levels  of  interactivity  and  feedback  received  via 
electronic  message  were  two  other  course  design  features  that  positively  influenced  their 
motives  to  complete  the  e-learning  course.  Interestingly  though,  one- third  of  the 
respondents  identified  “lack  of  interactivity  with  instructors  and  other  students”  as  the 
most  unappealing  feature  of  e- learning,  but  messages  received  from  instructors  did  not 
significantly  influence  their  desire  to  complete  the  course.  These  finding  contradict  each 
other,  but  response  data  revealed  that  the  probability  of  dropping  out  was  higher  when 
messages  were  received  from  the  instructor.  A  higher  percentage  of  dropouts  indicated 
that  the  messages  received  from  their  instructors  were  either  related  to  their  course 
performance  or  warnings  that  they  were  running  out  of  time  to  complete  the  course.  This 
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leads  to  the  belief  that  they  were  either  doing  well  and  other  factors  caused  them  to  drop 
out,  or  doing  poorly  and  decided  to  drop  out  based  on  the  feedback  received  from  their 
instructor. 

Findings  from  research  question  two  suggest  that  e-learning  course  designers 
should  continue  to  enhance  interactivity  features,  but  shorten  module  lengths.  In 
addition,  it  appears  that  “feedback  matters,”  and  that  providing  some  positive  feedback 
may  improve  completion  rates. 

Research  Question  3  Discussion 

Research  question  three  was  the  fundamental  question  of  this  study.  Its  objective 
was  to  determine  what  motivational  factors  influence  e-learning  course  completion  rates. 
Recall,  motivational  factors  are  classified  as  either  distal  or  proximal.  The  distal  theories 
of  motivation  emphasize  processes  that  affect  goal  choice  and  intended  future  effort.  The 
proximal  theories  of  motivation  emphasize  processes  that  control  the  initiation  and 
execution  of  actions  during  task  engagement  (Kanfer,  1990).  Based  on  the  results  of  the 
six  hypotheses  (H2a,  H2h,  H2c,  H3,  H4,  and  H^)  used  to  test  this  question,  it  was  determined 
that,  within  the  distal  system  of  motivation,  certain  types  of  intrinsic  and  extrinsic  factors 
along  with  goal  importance  and  commitment  significantly  influenced  completion  rates. 
Self-efficacy  was  the  only  proximal  construct  that  proved  statistically  reliable  in  terms  of 
influencing  completion  rates. 

Test  results  on  intrinsic  motivation  indicate  that  the  majority  of  the  respondents 
enrolled  for  intrinsically  motivated  reasons  (i.e.  gain  knowledge,  improve  job 
performance,  any  time  learning,  etc.),  but  there  were  only  minor  differences  on  intrinsic 
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motives  between  the  complete  and  dropped  groups.  Though  not  significant,  the  majority 
of  the  respondents  that  dropped  out  indicated  they  had  a  need  for  competence.  The  self- 
determination  factors  of  “any  time”  learning  and  “ability  to  fit  into  schedule”  had  a 
positive  influence  on  completion  rates.  Likewise,  there  was  a  positive  relationship 
between  respondents  that  selected  the  extrinsic  reward  of  “degree/certificate  credits”  and 
course  completion  rates.  Goal  intensity,  or  the  strength  of  the  goal  in  relation  to  goal 
importance  and  goal  commitment  (Lock,  1968),  was  the  only  other  distal  factor  that 
significantly  influenced  completion  rates.  Results  indicated  that  respondents  completed 
their  courses  more  often  when  the  course  had  the  goal  intensity  characteristics  of  being 
“required”  for  job  purposes,  or  recommended  by  a  supervisor. 

The  respondents’  motivational  tendencies  were  also  tested  to  determine  if 
achievement  motives  significantly  influenced  course  completion  rates.  Results  reveal 
that  they  did  not  significantly  influence  completion  rates.  It  is  believed,  however,  that 
self-serving  bias  contaminated  these  results.  The  survey  question  used  to  test  this 
hypothesis  asked  respondents  to  rate  themselves  in  terms  of  their  motivation  to  achieve 
goals.  The  majority  of  the  respondents  rated  themselves  as  being  both  “highly 
motivated”  and  “success-oriented,”  which  is  consistent  with  self-serving  bias  theory 
(Myers,  1996). 

Self-efficacy  (confidence)  was  the  only  proximal  motivational  factor  that 
significantly  influenced  completion  rates.  Findings  indicate  that  most  of  the  respondents 
had  high  self-efficacy  with  e- learning  that  lasted  from  the  start  to  the  end  of  the  course. 
Respondents  indicating  their  self-efficacy  “got  worse”  as  they  progressed  through  the  e- 
learning  course  had  higher  drop  out  rates  than  those  indicating  “no  change”  or  “got 
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better.”  These  findings  suggest  that  a  higher  percentage  of  e- learners  drop  out  as  self- 
efficacy  beliefs  about  their  ability  to  complete  the  course  decrease.  It  was  also  found  that 
low  initial  confidence  negatively  influenced  a  respondent’s  desire  to  complete  the  course. 

Though  many  of  the  hypotheses  used  to  test  the  distal  and  proximal  theories  of 
motivation  were  not  statistically  supported,  findings  suggest  motivational  constructs  may 
help  to  determine  the  likelihood  that  students  will  complete  or  drop  out  of  e-  learning 
courses.  Therefore,  e-  learning  course  instructors,  designers,  and  administrators  would  be 
well  advised  to  find  ways  to  design  motivational  features  into  the  courses  as  well  as  their 
lesson  plans  and  course  curriculum  that  enhance  such  aspects  as  “any  time”  learning,  the 
salience  of  extrinsic  rewards,  goal  intensity,  and  a  student’s  self-efficacy. 

Implications 

While  e-  learning  courses  have  considerably  improved  over  the  past  few  years 
(Rosenberg,  2001),  it  is  apparent  from  the  responses  that  there  is  still  room  for  design 
improvements.  In  addition,  more  emphasis  can  be  placed  on  providing  the  right  and 
necessary  environmental  support  to  e-  learners.  From  a  theoretical  standpoint,  the 
integrative  motivational  approach  to  e- learning  demonstrated  that  external  factors,  along 
with  e-learning  course  design,  significantly  affect  a  user’s  desire  to  invest  time,  talent, 
and  energy  into  e- learning.  This  finding  suggests  that  motivational  theory  can  be  used  to 
predict  and  explain  the  probability  either  that  a  particular  person  will  complete  an  e- 
leaming  course,  or  that  a  particular  e-  learning  course  will  be  completed. 

Practitioners  who  desire  to  produce  e-  learning  courses  that  have  a  higher 
probability  of  being  completed  can  use  this  integrative  approach  to  motivation  to  assist 
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them  in  the  development  process.  Academics  can  also  use  these  findings  to  get  a  better 
understanding  of  their  students  and  develop  useful  education,  lesson,  and  curricula  plans 
designed  to  initiate  and  sustained  goal- directed  behavior.  Specifically,  e-learning  course 
instructors,  administrators,  and  designers,  should  infer  from  the  findings  that 
improvements  to  e-  learning  course  completion  rates,  which  is  chiefly  determined  by 
persistent  behavior,  will  come  about  as  a  result  of  their  increased  understanding  of  how 
course  designs,  environmental  support,  and  feedback  actions  both  motivate  and  de¬ 
motivate  e-  learners. 

The  Air  Force  has  indicated  its  desire  to  harness  the  capabilities  of  e- learning 
through  its  continual  investment  and  implementation  of  e-  learning  technology.  A  pivotal 
factor  in  maximizing  the  power  of  this  technology  becomes  the  ability  to  create  courses 
that  preserve  an  e- learner’s  desire  to  persist  until  completion.  Now  that  external  factors, 
course  design  characteristics,  and  motivational  constructs  that  influence  the  desire  to 
persist  have  been  identified,  practitioners  and  academics  may  be  able  to  design  better  e- 
leaming  courses  that  optimize  their  use. 

Limitations 

The  most  notable  limitation  of  this  study  was  the  use  of  unproven,  untested 
methods  to  measure  the  course  feedback  characteristics.  Due  to  the  time  constraints 
placed  on  this  research,  empirical  methods  were  not  found  to  assist  in  this  area. 

Therefore,  feedback  characteristic  measures  were  subjective  ones  made  from  the 
observations  of  one  person.  Furthermore,  the  method  used  to  measure  readability  (i.e.  the 
Flesch  Reading  Ease  Score)  was  designed  for  black  text  on  paper,  not  text  located  on  a 
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computer  screen  that  often  appeared  in  different  colors  and  even  sometimes  moved 
around  the  screen.  This  study  could  have  benefited  from  seeking  and  implementing  a 
proven  and  well- tested  method  to  measure  feedback  and  readability  characteristics. 

The  second  limitation  of  the  study  is  that  it  was  assumed  that  each  of  the 
respondents  had  the  same  opportunity  to  engage  with  their  e-  learning  course  that 
everyone  else  in  the  study  had.  For  instance,  it  can  only  be  assumed  that  each  respondent 
had  easy  access  to  a  computer.  In  reality,  the  e-  learning  experience  can  be  quite  different 
depending  on  many  factors  not  readily  assessed  or  discussed  during  this  study.  Instructor 
and  location  differences  are  two  of  a  possible  many.  In  addition,  events  like  the 
September  1 1  World  Trade  Center  Disaster  may  have  significantly  changed  respondents’ 
environment  or  psyche  in  a  way  that  highly  influenced  their  desires  to  persist  at  e- 
leaming. 

The  fact  that  responses  are  self-report  is  a  limitation  that  could  not  be  avoided. 
Such  factors  as  self-serving  bias  (Myers,  1996)  could  have  inflated  responses,  or  negative 
experiences  could  have  caused  responses  to  be  understated.  Both  cases  taint  results. 

Another  limitation  concerns  the  fact  that  no  pilot  test  was  conducted  before  the  E- 
Leaming  Course  Questionnaire  was  deployed  to  the  respondents.  Though  it  was 
validated  by  subject  matter  experts,  a  pilot  test  could  have  helped  eliminate  some 
potentially  ambiguous  choices  contained  within  a  few  of  the  items.  This  may  have 
helped  produce  even  better  results. 

A  final  limitation  is  that  the  subject  pool  consisted  of  only  Air  Force  members. 
Though  these  members  were  located  worldwide,  they  still  belonged  to  the  same 
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population — the  Air  Force.  A  similar  study  looking  at  other  populations  would  help 
generalize  the  study. 

Future  Research 

Several  opportunities  exist  for  future  research  of  this  topic.  For  starters,  the 
integrative  motivational  model  was  developed  but  not  validated.  Future  research  could 
take  this  model,  test  it,  and  validate  its  usefulness  as  a  true  determinant  of  e-  learning 
course  completion  rates.  The  model  could  be  tested  and  validated  under  field  conditions 
using  pretests,  placebo  groups,  and  possibly  a  manipulated  environment.  Additionally, 
the  same  study  could  be  separately  conducted  on  both  college  and  corporate  e-  learning 
students.  Results  could  then  be  compared  to  see  if  the  same  motivational  factors  are 
deemed  significant  across  the  different  domains. 

In  retrospect,  those  respondents  who  had  to  retake  or  extend  their  courses  should 
have  been  analyzed  separately  from  those  that  completed  or  dropped  their  course  on  the 
first  try.  From  just  looking  at  percentages,  29  of  the  33  (87.9%)  that  dropped  stated  they 
either  retook  the  course  or  asked  for  an  extension.  Contingency  analyses  could  have  been 
conducted  comparing  the  completed  group  to  a  combined  group  of  dropped,  retakes,  and 
extensions.  Comparisons  could  then  be  made  to  the  previous  comparisons  of  the 
completed  and  dropped  groups.  This  could  present  itself  as  a  future  research  opportunity 
worth  investigating. 
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Conclusion 


The  results  of  this  study  supported  the  belief  that  certain  factors  such  as,  “any 
time”  learning,  goal  intensity,  and  self-efficacy  beliefs  initiate  and  direct  behavior,  as 
well  as  determine  how  much  time  and  effort  students  will  devote  to  the  e-  learning  course. 
Other  factors  like  slow  or  choppy  system  response,  network  outages,  environmental 
support,  average  time  per  module,  and  interactivity  level  appear  to  directly  and  indirectly 
influence  an  e-  learners  desire  to  persist  at  e-  learning.  Though  it  is  impossible  to  predict 
the  occurrence  of  external  factors  or  motivated  behavior,  practitioners  and  academics 
alike  can  benefit  from  knowing  which  of  these  factors  are  most  likely  to  influence  course 
completion  rates.  They  can  then  develop  methods  and  design  new  courses  that  seek  to 
improve  e-  learning  course  completion  rates,  thus  optimizing  its  growing  potential. 

In  summary,  this  study  used  an  integrative  approach  to  motivation,  in  conjunction 
with  the  methodological  steps  of  interviewing,  observation,  critique  analysis,  and  a 
questionnaire,  to  identify  and  measure  those  motivational  factors  that  significantly 
influenced  e- learning  course  completion  rates.  There  are  still  many  questions  to  be 
answered,  but  a  step  forward  has  been  made  by  a  study  that  has  shown  “going  the 
distance”  in  an  e-  learning  environment  is  not  as  straightforward  as  one  may  think. 
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Appendix  A:  Bloom’s  Taxonomy 


1.  KNOWLEDGE 

•  observation  and  recall  of  information 

•  knowledge  of  dates,  events,  places 

•  knowledge  of  major  ideas 

•  mastery  of  subject  matter 

•  Question  Cues: 

list,  define,  tell,  describe,  identify,  show,  label,  collect,  examine,  tabulate,  quote, 
name,  who,  when,  where,  etc. 


2.  COMPREHENSION 

•  understanding  information 

•  grasp  meaning 

•  translate  knowledge  into  new  context 

•  interpret  facts,  compare,  contrast 

•  order,  group,  infer  causes 

•  predict  consequences 

•  Question  Cues: 

summarize,  describe,  interpret,  contrast,  predict,  associate,  distinguish,  estimate, 
differentiate,  discuss,  extend 


3.  APPLICATION 

•  use  information 

•  use  methods,  concepts,  theories  in  new  situations 

•  solve  problems  using  required  skills  or  knowledge 

•  Questions  Cues: 

apply,  demonstrate,  calculate,  complete,  illustrate,  show,  solve,  examine,  modify, 
relate,  change,  classify,  experiment,  discover 
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4.  ANALYSIS 


•  seeing  patterns 

•  organization  of  parts 

•  recognition  of  hidden  meanings 

•  identification  of  components 

•  Question  Cues: 

analyze,  separate,  order,  explain,  connect,  classify,  arrange,  divide,  compare, 
select,  explain,  infer 


5.  SYNTHESIS 

•  use  old  ideas  to  create  new  ones 

•  generalize  from  given  facts 

•  relate  knowledge  from  several  areas 

•  predict,  draw  conclusions 

•  Question  Cues: 

combine,  integrate,  modify,  rearrange,  substitute,  plan,  create,  design,  invent, 
what  if?,  compose,  formulate,  prepare,  generalize,  rewrite 


6.  EVALUATION 

•  compare  and  discriminate  between  ideas 

•  assess  value  of  theories,  presentations 

•  make  choices  based  on  reasoned  argument 

•  verify  value  of  evidence 

•  recognize  subjectivity 

•  Question  Cues 

assess,  decide,  rank,  grade,  test,  measure,  recommend,  convince,  select,  judge, 
explain,  discriminate,  support,  conclude,  compare,  summarize 
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Appendix  B:  Content  Analysis  of  E-Learning  Course  Critiques 


Most  frequently  made  comments  on  e- learning  course  critiques. 


Possible  Reasons  Why  Students 

Complete  (frequency) 

Possible  Reasons  Why  Students 

Drop  Out  (frequency) 

“I  found  a  wealth  of  information  in  the 

“1  experienced  a  lot  of  technical 

materiaf’  (34) 

difficulty”  (53) 

“The  course  was  of  value  to  my  future”  (30) 

“It  helped  me  understand  my  job  position” 

“Difficulty  completing  with  current 

workload  and  distractions”  (45) 

“Too  many  content  (e.g.,  typographical) 

(28) 

errors”  (37) 

“I  had  no  problems  at  all”  (24) 

“Some  information  was  out  of  date”  (32) 

“Functionality  was  easy”  (19) 

“Many  links  did  not  work”  (30) 

“Could  learn  at  your  own  pace”  (18) 

“Could  take  the  course  from  work  or  home” 

(18) 

“Liked  the  interactive  exercises”  (11) 

“Low  amounts  of  feedback  and 

interaction”  (28) 

“Poor  or  boring  content”  (26) 

“1  was  sent  TDY  in  middle  of  course”  (9) 
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Appendix  C:  Initial  Email  Message  Sent  to  Respondents 


The  Air  Force  Institute  of  Technology  (AFIT)  is  conducting  research  on  ways  to 
improve  e-  learning.  It  has  been  revealed  that  you  were  recently  enrolled  in  the  (tar2et 
course  was  input  here)  e-learning  course  administered  by  the  Defense  Acquisition 
University  (DAU)  Virtual  Campus  [or  AFIT/LSB  Virtual  Schoolhouse,  depending  on 
target  course].  Therefore,  it  would  be  greatly  appreciated  if  you  participate  in  this  study 
by  filling  out  the  questionnaire  located  at  the  following  link: 

http://en.afit.edu/env/eleaming.  It  should  take  no  more  than  10  -  15  minutes  to  complete. 

The  purpose  of  the  study  is  to  provide  e-learning  instructors,  administrators,  and 
designers  specific  information  on  how  to  develop  e- learning  courses  that  better  suit  your 
needs.  The  study  was  reviewed  and  approved  by  the  AFPC  Survey  Branch  (Reference: 
USAF  Survey  Control  Number  01-120).  However,  if  you  have  any  questions  or 
concerns,  please  feel  free  to  reply  to  this  email.  Please  note  you  are  free  to  terminate 
your  participation  at  anytime. 

We  are  very  interested  in  your  responses  to  the  questionnaire  for  they  will  be  used 
to  build  much  better  and  more  useful  e-learning  courses.  Thank  you  in  advance  for  your 
participation. 

Just  click  on  the  above  link  to  begin. 

E-Learning  Research  Team 
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Appendix  D:  E-Learning  Course  Questionnaire 


E-Learning  Cours§  Questionnaire  (ECQ) 


Welcome  to  the  E-Learning  Course  Questionnaire  (ECQ^ 

Please  take  the  next  few  minutes  to  answer  the  following  series  of  statements 
regarding  the  e-  learning  course  you  recently  took 
(i.e.  the  one  referenced  in  the  e-mail). 

The  ECQ  provides  you  the  opportunity  to  give  e-  learning  instructors, 
administrators,  and  designers  feedback  on  how  to  develop  better  e-  learning  courses. 
Your  response  to  the  ECQ  will  be  combined  with  the  responses  of  other  members  who 
have  taken  the  same  course,  as  well  as  compared  to  those  who  have  taken  other  e- 
learning  courses.  Results  will  be  provided  to  instructors,  administrators,  and  designers  of 

the  courses  in  question. 


Instructions: 

The  survey  will  first  ask  for  some  demographic  information. 

Several  steps  have  been  taken  to  protect  your  anonymity.  First,  you  will 
not  be  asked  to  provide  your  name,  age,  race,  gender,  or  unit  at  any  time. 
Second,  your  questionnaire  responses  will  be  entered  directly  in  to  a 
database  that  has  no  way  of  determining  from  whom  the  information  is 
being  sent. 

There  are  three  types  of  questions  in  this  survey:  1.  Check  all  that 
apply,  2.  Choose  the  best  answer,  and  3.  5-point  Likert  Scale.  For  the 
“check  all  that  apply”  questions,  select  ^  the  answers  you  feel 
adequately  described  your  experience.  For  the  “choose  the  best  answer” 
questions,  select  the  one  best  answer  that  described  your  experience.  And 
for  the  “Likert  Scale”  questions,  select  one  answer  between  Strongly 
Disagree  (1)  and  Strongly  Agree  (5).  Please  read  and  answer  each 
statement  before  submitting  your  results.  Also, 

USE  YOUR  BROWSER’S  'BACK"  BUTTON  TO  RETURN  TO 

PREVIOUS  PAGES 

The  ECQ  should  take  10  -  15  minutes  to  complete. 
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bemographic  Information 

Please  enter  the  following  demographic  information: 


Rank/Grade: 


Marital  Status:  Married:  ^  Single:  ^  Children:  Yes:  ^  No:  ^ 


Please  indicate  the  e-learning  course  that  you  were  enrolled  in? 

▼ 


Please  Select  a  Course 


pi  p 

Have  you  completed  the  course?  Yes:  No: 


p  p 

Did  you  need  an  extension  at  any  time  while  taking  the  course?  Yes:  No: 


P  p 

Did  you  have  to  retake  the  course  for  any  reason?  Yes:  No:  ^ 


How  many  e-learning  (or  web-based)  courses  had  you  taken  PRIOR  to  the  one  in 
question?  I”  *Please  use  numbers  only 


101 


ECQ 


Please  read  and  answer  each  statement  carefully. 


Why  did  you  take  the  Modification  Management  (SAS030V)  e-learning  course? 

(Check  all  that  apply) 

^  Job  requirement 
^  To  gain  knowledge 
^  Improve  job  performance 
^  Supervisor's  recommendation 
^  Promotion  opportunity 

^  Accumulate  degree/certificate  credit,  continuous  learning  points,  etc. 

Out  of  curiosity 

^  Get  some  specific  type  of  information 

r  I 

Other,  please  specify  I 


2. 


Completing  this  Modification  Management  (SAS030V)  course  was  important  to 
me. 


Strongly  Disagree 

C 


Disagree  Indifferent  Agree  Strongly  Agree 

c  c  c  c 


page  1  of  10 


Once  I  enrolled  in  the  Modification  Management  (SAS030V)  my  initial  goal  (or 
intention)  was  to  complete  it. 

Strongly  Disagree  Disagree  Indifferent  Agree  Strongly  Agree 
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From  the  beginning,  I  planned  to  give  the  Modification  Management 
(SAS030V)  my  best  possible  effort. 

Strongly  Disagree  Disagree  Indifferent  Agree  Strongly  Agree 

C  C  C  C  G 

As  I  started  the  Modification  Management  (SAS030V)  course,  I  was  confident  I 
would  complete  it. 

Strongly  Disagree  Disagree  Indifferent  Agree  Strongly  Agree 

C  C  C  C  C 


Did  this  confidence  change  as  you  progressed  through  the  Modification 
Management  (SAS030V)  course?  (Choose  best  answer) 
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7.  How  do  you  view  yourself  as  a  worker/learner?  (Choose  best  answer) 

^  Highly  motivated 

G 

Somewhat  motivated 

G 

Can  motivate  myself  when  needed 

G 

^  Motivated  if  prodded  or  rewarded 
^  Not  motivated  at  all 
^  Other,  please  specify 

g  In  terms  of  working  towards  goals,  do  you  consider  yourself  as:  (Choose  best 
answer) 

G 

Having  stronger  motives  to  achieve  success  (i.e.  try  to  do  better  than  others; 
demonstrate  superior  ability,  etc.) 

G 

Having  stronger  motives  to  avoid  failure  (i.e.  worry  about  doing  poorly;  afraid 
you  may  ask  a  "dumb"  question,  etc.) 
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9. 


In  what  ways,  if  any,  did  you  find  e-learning  appealing?  (Check  all  that  apply) 
^  Convenience  of  “any  time”  learning 
^  Convenience  of  “anywhere”  learning 
^  Could  work/leam  independently 
^  Could  work/learn  at  own  pace 
^  Ability  to  fit  into  schedule 
^  Other,  please  specify  I 
^  I  DID  NOT  find  e-  learning  appealing  at  all 


10.  In  what  ways,  if  any,  did  you  find  e-learning  unappealing?  (Check  all  that  apply) 
^  Lack  of  interactivity  with  instructor  and  other  students 
^  Not  enough  "hands-on"  exercises  and  activities 
Lack  of  personalized  feedback 
^  Uncompelling,  static  nature  of  course  content 
Browser/connectivity  problems 

Slow  or  choppy  system  response  times  (i.e.  time  it  takes  computer  to  respond  to 
a  user  command) 

^  Lack  of  course  instruction  and  guidance 
^  Other,  please  specify  I 
^  I  DID  NOT  find  e-  learning  unappealing  at  all 


page  4  of  10 
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11.  Until  you  stopped  working  on  the  Modification  Management  (SAS030V)  course 
(i.e.  withdrew  or  completed),  what  caused  or  triggered  you  to  persist?  (Choose 
best  answer) 

My  original  intentions  to  complete  the  course 

P 

Peer  pressure  (i.e.  competition,  people  around  me  liked  e- learning,  etc.) 

P 

Supervisor  expectations 

P 

Course  was  easy 

P 

Course  was  short 
^  Thoughts  of  accomplishment 

^  Had  not  gotten  the  information  I  needed  or  desired  yet 

p 

Feedback  that  I  was  doing  well  in  the  course 

I 

Other,  please  specify  I 


12.  What  caused  or  triggered  you  to  stop  working  on  the  Modification  Management 
(SAS030V)  course?  (Choose  best  answer) 

*Note:  If  you  completed  the  Modification  Management  (SAS030V)  course,  please 
select  the  first  option. 

P 

1  completed  the  course 

P 

^  Too  many  distractions 

P 

Course  was  too  difficult 

P 

Course  was  too  long 

^  Peer  pressure  (i.e.  co-workers  thought  I  was  “slacking  off’,  etc.) 

^  Had  to  leave  on  a  business  trip  (i.e.  TOY) 

P 

Network/Hardware  problems 

^  The  content  was  not  what  I  expected 

P 

Other  demands  became  more  important  (i.e.  job,  personal) 

P 

Feedback  that  I  was  not  doing  well  in  the  course 

P 

I  exceeded  the  course’s  time  constraint 

P 

I  got  the  information  I  needed 

I  ' 

Other,  please  specify  I 
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Even  when  I  scheduled  to  do  e-learning,  I  felt  like  something  would  happen  to 
make  me  change  my  plans. 

Strongly  Disagree  Disagree  Indifferent  Agree  Strongly  Agree 

C  C  C  C  C 


14.  When  and  where  did  you  typically  take  this  course?  (Choose  best  answer) 
On  the  job  during  regular  business  hours 
On  the  job  during  non-business  hours 

At  home  during  business  hours 

P 

^  At  home  during  non-business  hours 

P 

During  the  weekend 

P 

^  While  on  a  business  trip  (i.e.  TDY) 

P 

^  Other,  please  specify 
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15. 


What  distractions,  if  any,  did  you  encounter  while  taking  the  Modification 
Management  (SAS030V)  course?  (Check  all  that  apply.  For  those  checked,  choose 
how  often  you  were  distracted  from  the  drop-down  menu  next  to  the  item) 

^  Noise  (i.e.  phone,  office  chatter,  television,  etc.)  - 


Job-related  demands  (i.e.  meetings,  deadlines,  requests,  etc.)  - 


^  Personal  demands  (i.e.  family,  friends,  clubs,  etc.)  - 
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r 

^  how  often... 

Slow  re^^pon^ie^;  - 

d 

r 

Hardware/Software  problems  -1  - 

-  i 

r 

Other,  please  specify  1 

1  how  often... 

d 

r 

I  was  NOT  distracted  at  all 

16. 


The  distractions  I  encountered  hindered  my  desire  to  persist  at  e-learning. 

Strongly  Disagree  Disagree  Indifferent  Agree  Strongly  Agree 

c  c  c  c  c 
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17.  I  was  unable  to  complete  the  Modification  Management  (SAS030V)  course 
because  of  distractions  I  encountered. 

Strongly  Disagree  Disagree  Indifferent  Agree  Strongly  Agree 

c  c  c  c  c 


18. 


I  had  a  lot  of  support  (i.e.  work,  family,  instructor,  peers,  etc.)  in  terms  of  being 
allowed  time  to  devote  attention  to  the  Modification  Management  (SAS030V) 
course. 

Strongly  Disagree  Disagree  Indifferent  Agree  Strongly  Agree 


C 


c  c 


c  c 


.  g  What  type  of  feedback  did  you  receive  while  engaged  with  e-learning?  (Check 
all  that  apply) 

^  Electronic  messages  from  the  course  on  results  of  quizzes  and  exercises 
^  Instructor  or  administrator  messages  on  results  of  quizzes  and  exercises 
^  Electronic  messages  related  to  hardware/software  issues 
^  Instructor  or  administrator  messages  related  to  hardware/software  issues 
^  Electronic  messages  related  to  your  overall  course  performance 
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Messages  from  an  instructor  or  administrator  related  to  your  course  performance 
Messages  received  as  a  result  of  questions  you  asked 
^  Other,  please  specify  I 
^  I  received  NO  feedback 
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20.  Receiving  feedback  is  important  to  me  while  I  am  engaged  with  e-learning. 

Strongly  Disagree  Disagree  Indifferent  Agree  Strongly  Agree 

c  c  c  c  c 


21.  I  believe  I  received  a  sufficient  amount  of  feedback  for  the  Modification 
Management  (SAS030V)  course  I  was  taking. 

Strongly  Disagree  Disagree  Indifferent  Agree  Strongly  Agree 

c  c  c  c  c 


22. 


The  feedback  I  did 

Strongly  Disagree 

C 


receive  was  timely  in  terms  of  how  long  it  took  to  receive  it. 

Disagree  Indifferent  Agree  Strongly  Agree 

c  c  c  c 
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22  I  was  able  to  use  the  feedback  I  received  to  properly  assess  my  progress  in  the 
Modification  Management  (SAS030V)  course. 

Strongly  Disagree  Disagree  Indifferent  Agree  Strongly  Agree 
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Do  you  have  any  additional  comments  you  would  like  to  add? 
Please  write  any  comments  you  have  below. 


Or  email  >our  comments  to  elearning@alit.edu 


This  completes  the  ECQ! 

Please  hit  the  “Finish"  button  below  to  submit  your  responses. 

THANK  YOU  FOR  YOUR  PARTICIPATION! 


page  10  of  10 


tciMtion, 


the  beflinning. 


Feel  free  to  email  us  at  elearning@afit.edu 


109 


Appendix  E:  Follow-up  Email  Message  Sent  to  Respondents 


Last  week  you  were  sent  an  email  requesting  you  to  fill  out  an  E-Learning 
Questionnaire  regarding  the  (target  course  was  input  here)  e-leaming  course  you  were 
enrolled  in.  If  you  filled  out  the  questionnaire,  we  thank  you  for  your  participation  and 
you  may  delete  this  email  if  you  wish. 

If  you  chose  not  to  participate,  we  urge  you  to  reconsider  and  take  the  next  few 
minutes  to  complete  it.  We  realize  that  some  of  you  encountered  problems  when  trying 
to  complete  the  survey  the  first  time.  Most  of  these  problems  have  been  corrected. 
However,  there  is  still  a  problem  with  the  Netscape  Web  browser.  Those  of  you  who 
have  Netscape  as  your  default  Web  browser  will  not  be  able  to  see  the  drop-down  box 
that  goes  with  the  statement  "Please  indicate  the  e-  learning  course  that  you  were  enrolled 
in"  (located  on  the  Demographic  Information  page).  The  only  way  around  this  problem 
is  use  Internet  Explorer.  Here  is  how  you  do  it: 

1)  Open  Internet  Explorer  (IE) 

2)  Cut  and  Paste  the  e-leaming  survey  hyperlink  into  IE 

3)  Then  start  and  take  the  survey 

Again,  your  responses  are  vital  to  our  efforts  to  improve  future  e-leaming  courses. 
So  please  click  on  the  hyperlink  below  and  join  us  in  making  e-leaming  a  better  and  more 
useful  tool  to  educate  and  train  our  workforce.  And  remember,  you  are  free  to  terminate 
your  participation  at  any  time. 

E-Learning  Research  Team 

http  ://en.  afit.edu/ env/eleaming 

Reply  to  this  email  if  you  have  questions  or  concerns 


no 
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